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UNIX® System throughout the computer industry. The author describes the 
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system (the kernel) and their relationship to the programmer interface. 

Among its key features, the book: 

• describes the outline of the kernel architecture 

• introduces the system buffer cache mechanism 

• includes data structures and algorithms used internally by the file system 
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PREFACE 



The UNIX system was first described in a 1974 paper in the Communications of 
the ACM [Thompson 74] by Ken Thompson and Dennis Ritchie. Since that time, 
it has become increasingly widespread and popular throughout the computer 
industry where more and more vendors are offering support for it on their 
machines. It is especially popular in universities where it is frequently used for 
operating systems research and case studies. 

Many books and papers have described parts of the system, among them, two 
special issues of the Bell System Technical Journal in 1978 [BSTJ 78] and 1984 
[BLTJ 84]. Many books describe the user level interface, particularly how to use 
electronic mail, how to prepare documents, or how to use the command interpreter 
called the shell; some books such as The UNIX Programming Environment 
[Kernighan 84] and Advanced UNIX Programming [Rochkind 85] describe the 
programming interface. This book describes the internal algorithms and structures 
that form the basis of the operating system (called the kernel) and their 
relationship to the programmer interface. It is thus applicable to several 
environments. First, it can be used as a textbook for an operating systems course 
at either the advanced undergraduate or first-year graduate level. It is most 
beneficial to reference the system source code when using the book, but the book 
can be read independently, too. Second, system programmers can use the book as a 
reference to gain better understanding of how the kernel works and to compare 
algorithms used in the UNIX system to algorithms used in other operating systems. 
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Finally, programmers on UNIX systems can gain a deeper understanding of how 
their programs interact with the system and thereby code more-efficient, 
sophisticated programs. 

The material and organization for the book grew out of a course that I prepared 
and taught at AT&T Bell Laboratories during 1983 and 1984. While the course 
centered on reading the source code for the system, I found that understanding the 
code was easier once the concepts of the algorithms had been mastered. I have 
attempted to keep the descriptions of algorithms in this book as simple as possible, 
reflecting in a small way the simplicity and elegance of the system it describes. 
Thus, the book is not a line-by-line rendition of the system written in English; it is 
a description of the general flow of the various algorithms, and most important, a 
description of how they interact with each other. Algorithms are presented in a C- 
like pseudo-code to aid the reader in understanding the natural language 
description, and their names correspond to the procedure names in the kernel. 
Figures depict the relationship between various data structures as the system 
manipulates them. In later chapters, small C programs illustrate many system 
concepts as they manifest themselves to users. In the interests of space and clarity, 
these examples do not usually check for error conditions, something that should 
always be done when writing programs. I have run them on System V; except for 
programs that exercise features specific to System V, they should run on other 
versions of the system, too. 

Many exercises originally prepared for the course have been included at the end 
of each chapter, and they are a key part of the book. Some exercises are 
straightforward, designed to illustrate concepts brought out in the text. Others are 
more difficult, designed to help the reader understand the system at a deeper level. 
Finally, some are exploratory in nature, designed for investigation as a research 
problem. Difficult exercises are marked with asterisks. 

The system description is based on UNIX System V Release 2 supported by 
AT&T, with some new features from Release 3. This is the system with which I 
am most familiar, but I have tried to portray interesting contributions of other 
variations to the operating system, particularly those of Berkeley Software 
Distribution (BSD). I have avoided issues that assume particular hardware 
characteristics, trying to cover the kernel-hardware interface in general terms and 
ignoring particular machine idiosyncrasies. Where machine-specific issues are 
important to understand implementation of the kernel, however, I delve into the 
relevant detail. At the very least, examination of these topics will highlight the 
parts of the operating system that are the most machine dependent. 

The reader must have programming experience with a high-level language and, 
preferably, with an assembly language as a prerequisite for understanding this 
book. It is recommended that the reader have experience working with the UNIX 
system and that the reader knows the C language [Kernighan 78]. However, I 
have attempted to write this book in such a way that the reader should still be able 
to absorb the material without such background. The appendix contains a 
simplified description of the system calls, sufficient to understand the presentation 
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in the book, but not a complete reference manual. 

The book is organized as follows. Chapter 1 is the introduction, giving a brief, 
general description of system features as perceived by the user and describing the 
system structure. Chapter 2 describes the general outline of the kernel architecture 
and presents some basic concepts. The remainder of the book follows the outline 
presented by the system architecture, describing the various components in a 
building block fashion. It can be divided into three parts: the file system, process 
control, and advanced topics. The file system is presented first, because its concepts 
are easier than those for process control. Thus, Chapter 3 describes the system 
buffer cache mechanism that is the foundation of the file system. Chapter 4 
describes the data structures and algorithms used internally by the file system. 
These algorithms use the algorithms explained in Chapter 3 and take care of the 
internal bookkeeping needed for managing user files. Chapter 5 describes the 
system calls that provide the user interface to the file system; they use the 
algorithms in Chapter 4 to access user files. 

Chapter 6 turns to the control of processes. It defines the context of a process 
and investigates the internal kernel primitives that manipulate the process context, 
in particular, it considers the system call interface, interrupt handling, and the 
context switch. Chapter 7 presents the system calls that control the process 
context. Chapter 8 deals with process scheduling, and Chapter 9 covers memory 
management, including swapping and paging systems. 

Chapter 10 outlines general driver interfaces, with specific discussion of disk 
drivers and terminal drivers. Although devices are logically part of the file system, 
their discussion is deferred until here because of issues in process control that arise 
in terminal drivers. This chapter also acts as a bridge to the more advanced topics 
presented in the rest of the book. Chapter 1 1 covers interprocess communication 
and networking, including System V messages, shared memory and semaphores, 
and BSD sockets. Chapter 12 explains tightly coupled multiprocessor UNIX 
systems, and Chapter 13 investigates loosely coupled distributed systems. 

The material in the first nine chapters could be covered in a one-semester course 
on operating systems, and the material in the remaining chapters could be covered 
in advanced seminars with various projects being done in parallel. 

A few caveats must be made at this time. No attempt has been made to 
describe system performance in absolute terms, nor is there any attempt to suggest 
configuration parameters for a system installation. Such data is likely to vary 
according to machine type, hardware configuration, system version and 
implementation, and application mix. Similarly, I have made a conscious effort to 
avoid predicting future development of UNIX operating system features. 
Discussion of advanced topics does not imply a commitment by AT&T to provide 
particular features, nor should it even imply that particular areas are under 
investigation. 

It is my pleasure to acknowledge the assistance of many friends and colleagues 
who encouraged me while I wrote this book and provided constructive criticism of 
the manuscript. My deepest appreciation goes to Ian Johnstone, who suggested 
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that I write this book, gave me early encouragement, and reviewed the earliest 
draft of the first chapters. Ian taught me many tricks of the trade, and I will 
always be indebted to him. Doris Ryan also had a hand in encouraging me from 
the very beginning, and I will always appreciate her kindness and thoughtfulness. 
Dennis Ritchie freely answered numerous questions on the historical and technical 
background of the system. Many people gave freely of their time and energy to 
review drafts of the manuscript, and this book owes a lot to their detailed 
comments. They are Debby Bach, Doug Bayer, Lenny Brandwein, Steve Buroff, 
Tom Butler, Ron Gomes, Mesut G undue, Laura Israel, Dean Jagels, Keith 
Kelleman, Brian Kernighan, Bob Martin, Bob Mitze, Dave Nowitz, Michael 
Poppers, Marilyn Safran, Curt Schimmel, Zvi Spitz, Tom Vaden, Bill Weber, 
Larry Wehr, and Bob Zarrow. Mary Fruhstuck provided help in preparing the 
manuscript for typesetting. I would like to thank my management for their 
continued support throughout this project and my colleagues, for providing such a 
stimulating atmosphere and wonderful work environment at AT&T Bell 
Laboratories. John Wait and the staff at Prentice-Hall provided much valuable 
assitance and advice to get the book into its final form. Last, but not least, my 
wife, Debby, gave me lots of emotional support, without which I could never have 
succeeded. 
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GENERAL OVERVIEW 
OF THE SYSTEM 



The UNIX system has become quite popular since its inception in 1969, running on 
machines of varying processing power from microprocessors to mainframes and 
providing a common execution environment across them. The system is divided 
into two parts. The first part consists of programs and services that have made the 
UNIX system environment so popular; it is the part readily apparent to users, 
including such programs as the shell, mail, text processing packages, and source 
code control systems. The second part consists of the operating system that 
supports these programs and services. This book gives a detailed description of the 
operating system. It concentrates on a description of UNIX System V produced by 
AT&T but considers interesting features provided by other versions too. It 
examines the major data structures and algorithms used in the operating system 
that ultimately provide users with the standard user interface. 

This chapter provides an introduction to the UNIX system. It reviews its 
history and outlines the overall system structure. The next chapter gives a more 
detailed introduction to the operating system. 



1.1 HISTORY 

In 1965, Bell Telephone Laboratories joined an effort with the General Electric 
Company and Project MAC of the Massachusetts Institute of Technology to 
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develop a new operating system called Multics [Organick 72]. The goals of the 
Multics system were to provide simultaneous computer access to a large community 
of users, to supply ample computation power and data storage, and to allow users to 
share their data easily, if desired. Many people who later took part in the early 
development of the UNIX system participated in the Multics work at Bell 
Laboratories. Although a primitive version of the Multics system was running on a 
GE 645 computer by 1969, it did not provide the general service computing for 
which it was intended, nor was it clear when its development goals would be met 
Consequently, Bell Laboratories ended its participation in the project. 

With the end of their work on the Multics project, members of the Computing 
Science Research Center at Bell Laboratories were left without a “convenient 
interactive computing service” [Ritchie 84a]. In an attempt to improve their 
programming environment, Ken Thompson, Dennis Ritchie, and others sketched a 
paper design of a file system that later evolved into an early version of the UNIX 
file system. Thompson wrote programs that simulated the behavior of the proposed 
file system and of programs in a demand-paging environment, and he even encoded 
a simple kernel for the GE 645 computer. At the same time, he wrote a game 
program, “Space Travel,” in Fortran for a GECOS system (the Honeywell 635), 
but the program was unsatisfactory because it was difficult to control the “space 
ship” and the program was expensive to run. Thompson later found a little-used 
PDP-7 computer that provided good graphic display and cheap executing power. 
Programming “Space Travel” for the PDP-7 enabled Thompson to learn about the 
machine, but its environment for program development required cross-assembly of 
the program on the GECOS machine and carrying paper tape for input to the 
PDP-7. To create a better development environment, Thompson and Ritchie 
implemented their system design on the PDP-7, including an early version of the 
UNIX file system, the process subsystem, and a small set of utility programs. 
Eventually, the new system no longer needed the GECOS system as a development 
environment but could support itself. The new system was given the name UNIX, 
a pun on the name Multics coined by another member of the Computing Science 
Research Center, Brian Kernighan. 

Although this early version of the UNIX system held much promise, it could 
not realize its potential until it was used in a real project. Thus, while providing a 
text processing system for the patent department at Bell Laboratories, the UNIX 
system was moved to a PDP-11 in 1971. The system was characterized by its small 
size: 16K bytes for the system, 8K bytes for user programs, a disk of 512K bytes, 
and a limit of 64K bytes per file. After its early success, Thompson set out to 
implement a Fortran compiler for the new system, but instead came up with the 
language B, influenced by BCPL [Richards 69]. B was an interpretive language 
with the performance drawbacks implied by such languages, so Ritchie developed it 
into one he called C, allowing generation of machine code, declaration of data 
types, and definition of data structures. In 1973, the operating system was 
rewritten in C, an unheard of step at the time, but one that was to have tremendous 
impact on its acceptance among outside users. The number of installations at Bell 
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Laboratories grew to about 25, and a UNIX Systems Group was formed to provide 
internal support. 

At this time, AT&T could not market computer products because of a 1956 
Consent Decree it had signed with the Federal government, but it provided the 
UNIX system to universities who requested it for educational purposes. AT&T 
neither advertised, marketed, nor supported the system, in adherence to the terms 
of the Consent Decree. Nevertheless, the system’s popularity steadily increased. In 
1974, Thompson and Ritchie published a paper describing the UNIX system in the 
Communications of the ACM [Thompson 74], giving further impetus to its 
acceptance. By 1977, the number of UNIX system sites had grown to about 500, 
of which 125 were in universities. UNIX systems became popular in the operating 
telephone companies, providing a good environment for program development, 
network transaction operations services, and real-time services (via MERT 
[Lycklama 78a]). Licenses of UNIX systems were provided to commercial 
institutions as well as universities. In 1977, Interactive Systems Corporation 
became the first Value Added Reseller (VAR) 1 of a UNIX system, enhancing it 
for use in office automation environments. 1977 also marked the year that the 
UNIX system was first “ported” to a non- PDF machine (that is, made to run on 
another machine with few or no changes), the Interdata 8/32. 

With the growing popularity of microprocessors, other companies ported the 
UNIX system to new machines, but its simplicity and clarity tempted many 
developers to enhance it in their own way, resulting in several variants of the basic 
system. In the period from 1977 to 1982, Bell Laboratories combined several 
AT&T variants into a single system, known commercially as UNIX System III. 
Bell Laboratories later added several features to UNIX System III, calling the new 
product UNIX System V, 2 and AT&T announced official support for System V in 
January 1983. However, people at the University of California at Berkeley had 
developed a variant to the UNIX system, the most recent version of which is called 
4.3 BSD for VAX machines, providing some new, interesting features. This book 
will concentrate on the description of UNIX System V and will occasionally talk 
about features provided in the BSD system. 

By the beginning of 1984, there were about 100,000 UNIX system installations 
in the world, running on machines with a wide range of computing power from 
microprocessors to mainframes and on machines across different manufacturers’ 
product lines. No other operating system can make that claim. Several reasons 
have been suggested for the popularity and success of the UNIX system. 



1 . Value Added Resellers add specific applications to a computer system to satisfy a particular market. 
They market the applications rather than the operating system upon which they run. 

2. What happened to System IV? An internal version of the system evolved into System V. 
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• The system is written in a high-level language, making it easy to read, 
understand, change, and move to other machines. Ritchie estimates that the 
first system in C was 20 to 40 percent larger and slower because it was not 
written in assembly language, but the advantages of using a higher-level 
language far outweigh the disadvantages (see page 1965 of [Ritchie 78b]). 

• It has a simple user interface that has the power to provide the services that 
users want. 

• It provides primitives that permit complex programs to be built from simpler 
programs. 

• It uses a hierarchical file system that allows easy maintenance and efficient 
implementation. 

• It uses a consistent format for files, the byte stream, making application 
programs easier to write. 

• It provides a simple, consistent interface to peripheral devices. 

• It is a multi-user, multiprocess system; each user can execute several processes 
simultaneously. 

• It hides the machine architecture from the user, making it easier to write 
programs that run on different hardware implementations. 

The philosophy of simplicity and consistency underscores the UNIX system and 
accounts for many of the reasons cited above. 

Although the operating system and many of the command programs are written 
in C, UNIX systems support other languages, including Fortran, Basic, Pascal, 
Ada, Cobol, Lisp, and Prolog. The UNIX system can support any language that 
has a compiler or interpreter and a system interface that maps user requests for 
operating system services to the standard set of requests used on UNIX systems. 



1.2 SYSTEM STRUCTURE 

Figure 1.1 depicts the high-level architecture of the UNIX system. The hardware 
at the center of the diagram provides the operating system with basic services that 
will be described in Section 1.5. The operating system interacts directly 3 with the 
hardware, providing common services to programs and insulating them from 
hardware idiosyncrasies. Viewing the system as a set of layers, the operating 
system is commonly called the system kernel , or just the kernel, emphasizing its 



3. In some implementations of the UNIX system, the operating system interacts with a native operating 
system that, in turn, interacts with the underlying hardware and provides necessary services to the 
system. Such configurations allow installations to run other operating systems and their applications 
in parallel to the UNIX system. The classic example of such a configuration is the MERT system 
[Lycklama 78a] More recent configurations include implementations for IBM System/370 
computers [Felton 84] and for UNIVAC 1 100 Series computers [Bodenstab 84], 
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Figure 1.1. Architecture of UNIX Systems 



isolation from user programs. Because programs are independent of the underlying 
hardware, it is easy to move them between UNIX systems running on different 
hardware if the programs do not make assumptions about the underlying hardware. 
For instance, programs that assume the size of a machine word are more difficult to 
move to other machines than programs that do not make this assumption. 

Programs such as the shell and editors (ed and vi) shown in the outer layers 
interact with the kernel by invoking a well defined set of system calls. The system 
calls instruct the kernel to do various operations for the calling program and 
exchange data between the kernel and the program. Several programs shown in the 
figure are in standard system configurations and are known as commands , but 
private user programs may also exist in this layer as indicated by the program 
whose name is a.out , the standard name for executable files produced by the C 
compiler. Other application programs can build on top of lower-level programs, 
hence the existence of the outermost layer in the figure. For example, the standard 
C compiler, cc, is in the outermost layer of the figure: it invokes a C preprocessor, 
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two-pass compiler, assembler, and loader (link-editor), all separate lower-level 
programs. Although the figure depicts a two-level hierarchy of application 
programs, users can extend the hierarchy to whatever levels are appropriate. 
Indeed, the style of programming favored by the UNIX system encourages the 
combination of existing programs to accomplish a task. 

Many application subsystems and programs that provide a high-level view of the 
system such as the shell, editors, SCCS (Source Code Control System), and 
document preparation packages, have gradually become synonymous with the name 
“UNIX system.” However, they all use lower-level services ultimately provided by 
the kernel, and they avail themselves of these services via the set of system calls. 
There are about 64 system calls in System V, of which fewer than 32 are used 
frequently. They have simple options that make them easy to use but provide the 
user with a lot of power. The set of system calls and the internal algorithms that 
implement them form the body of the kernel, and the study of the UNIX operating 
system presented in this book reduces to a detailed study and analysis of the system 
calls and their interaction with one another. In short, the kernel provides the 
services upon which all application programs in the UNIX system rely, and it 
defines those services. This book will frequently use the terms “UNIX system,” 
“kernel,” or “system,” but the intent is to refer to the kernel of the UNIX 
operating system and should be clear in context. 



1.3 USER PERSPECTIVE 

This section briefly reviews high-level features of the UNIX system such as the file 
system, the processing environment, and building block primitives (for example, 
pipes). Later chapters will explore kernel support of these features in detail. 



1.3.1 The File System 

The UNIX file system is characterized by 

• a hierarchical structure, 

• consistent treatment of file data, 

• the ability to create and delete files, 

• dynamic growth of files, 

• the protection of file data, 

• the treatment of peripheral devices (such as terminals and tape units) as files. 

The file system is organized as a tree with a single root node called root (written 
“/”); every non-leaf node of the file system structure is a directory of files, and files 
at the leaf nodes of the tree are either directories, regular files , or special device 
files. The name of a file is given by a path name that describes how to locate the 
file in the file system hierarchy. A path name is a sequence of component names 
separated by slash characters; a component is a sequence of characters that 
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Figure 1.2. Sample File System Tree 



designates a file name that is uniquely contained in the previous (directory) 
component. A full path name starts with a slash character and specifies a file that 
can be found by starting at the file system root and traversing the file tree, 
following the branches that lead to successive component names of the path name. 
Thus, the path names “/etc/passwd”, “/bin/who”, and “/usr/src/cmd/who.c” 
designate files in the tree shown in Figure 1.2, but “/bin/passwd” and 
“/usr/src/date.c” do not. A path name does not have to start from root but can be 
designated relative to the current directory of an executing process, by omitting the 
initial slash in the path name. Thus, starting from directory “/dev”, the path name 
“ttyOl” designates the file whose full path name is “/dev/ttyOl”. 

Programs in the UNIX system have no knowledge of the internal format in 
which the kernel stores file data, treating the data as an unformatted stream of 
bytes. Programs may interpret the byte stream as they wish, but the interpretation 
has no bearing on how the operating system stores the data. Thus, the syntax of 
accessing the data in a file is defined by the system and is identical for all 
programs, but the semantics of the data are imposed by the program. For example, 
the text formatting program troff expects to find “new-line” characters at the end 
of each line of text, and the system accounting program acctcom expects to find 
fixed length records. Both programs use the same system services to access the 
data in the file as a byte stream, and internally, they parse the stream into a 
suitable format. If either program discovers that the format is incorrect, it is 
responsible for taking the appropriate action. 

Directories are like regular files in this respect; the system treats the data in a 
directory as a byte stream, but the data contains the names of the files in the 
directory in a predictable format so that the operating system and programs such as 




8 



GENERAL OVERVIEW OF THE SYSTEM 



Is (list the names and attributes of files) can discover the files in a directory. 

Permission to access a file is controlled by access permissions associated with 
the file. Access permissions can be set independently to control read, write, and 
execute permission for three classes of users: the file owner, a file group, and 
everyone else. Users may create files if directory access permissions allow it. The 
newly created files are leaf nodes of the file system directory structure. 

To the user, the UNIX system treats devices as if they were files. Devices, 
designated by special device files, occupy node positions in the file system directory 
structure. Programs access devices with the same syntax they use when accessing 
regular files; the semantics of reading and writing devices are to a large degree the 
same as reading and writing regular files. Devices are protected in the same way 
that regular files are protected: by proper setting of their (file) access permissions. 
Because device names look like the names of regular files and because the same 
operations work for devices and regular files, most programs do not have to know 
internally the types of files they manipulate. 

For example, consider the C program in Figure 1.3, which makes a new copy of 
an existing file. Suppose the name of the executable version of the program is 
copy. A user at a terminal invokes the program by typing 

copy oldfile newfile 

where oldfile is the name of the existing file and newfile is the name of the new file. 
The system invokes main , supplying arge as the number of parameters in the list 
argv, and initializing each member of the array argv to point to a user-supplied 
parameter. In the example above, arge is 3, argvfO] points to the character string 
copy (the program name is conventionally the Oth parameter), argv [l ] points to the 
character string oldfile , and argv[2] points to the character string newfile . The 
program then checks that it has been invoked with the proper number of 
parameters. If so, it invokes the open system call “read-only” for the file oldfile , 
and if the system call succeeds, invokes the creat system call to create newfile. The 
permission modes on the newly created file will be 0666 (octal), allowing all users 
access to the file for reading and writing. All system calls return —1 on failure; if 
the open or creat calls fail, the program prints a message and calls the exit system 
call with return status 1, terminating its execution and indicating that something 
went wrong. 

The open and creat system calls return an integer called a file descriptor , which 
the program uses for subsequent references to the files. The program then calls the 
subroutine copy , which goes into a loop, invoking the read system call to read a 
buffer’s worth of characters from the existing file, and invoking the write system 
call to write the data to the new file. The read system call returns the number of 
bytes read, returning 0 when it reaches the end of file. The program finishes the 
loop when it encounters the end of file, or when there is some error on the read 
system call (it does not check for write errors). Then it returns from copy and 
exits with return status 0, indicating that the program completed successfully. 
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#include <fcntl.h> 
char buffer{2048]; 

int version - 1; /• Chapter 2 explains this */ 

main (argc, argv) 
int argc; 
char *argv[]; 

I 

int fdold, fdnew; 

if (argc !“ 3) 

{ 

printfC’need 2 arguments for copy program\n"); 
cxit(l); 

} 

fdold — open(argv[ll, O RDONLY); /* open source file read only */ 

if (fdold 1) 

{ 

printf ("cannot open file %s\n", argyll]); 
exit(l); 

} 

fdnew — creat(argv{2], 06GG); /* create target file rw for all */ 

if (fdnew “ —1) 

{ 

printf ("cannot create file %s\n", argv(2]); 
exit(l); 

) 

copy (fdold, fdnew); 
exit(O); 

} 

copy (old, new) 

int old, new; 

l 

int count; 

while ((count — read(old, buffer, sizeof(buffer))) > 0) 
writc(ncw, buffer, count); 

} 



Figure 1.3. Program to Copy a File 



The program copies any files supplied to it as arguments, provided it has 
permission to open the existing file and permission to create the new file. The file 
can be a file of printable characters, such as the source code for the program, or it 
can contain unprintable characters, even the program itself. Thus, the two 
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invocations 

copy copy.c newcopy.c 
copy copy newcopy 

both work. The old file can also be a directory. For instance, 
copy . dircontents 

copies the contents of the current directory, denoted by the name to a regular 
file, “dircontents”; the data in the new file is identical, byte for byte, to the contents 
of the directory, but the file is a regular file. (The system call mknod creates a 
new directory.) Finally, either file can be a device special file. For example, 

copy /dev/tty terminalread 

reads the characters typed at the terminal (the special file Idev/tty is the user’s 
terminal) and copies them to the file terminalread , terminating only when the user 
types the character control-d. Similarly, 

copy /dev/tty /dev/tty 

reads characters typed at the terminal and copies them back. 



1.3.2 Processing Environment 

A program is an executable file, and a process is an instance of the program in 
execution. Many processes can execute simultaneously on UNIX systems (this 
feature is sometimes called multiprogramming or multitasking) with no logical limit 
to their number, and many instances of a program (such as copy) can exist 
simultaneously in the system. Various system calls allow processes to create new 
processes, terminate processes, synchronize stages of process execution, and control 
reaction to various events. Subject to their use of system calls, processes execute 
independently of each other. 

For example, a process executing the program in Figure 1.4 executes the fork 
system call to create a new process. The new process, called the child process, gets 
a 0 return value from fork and invokes execl to execute the program copy (the 
program in Figure 1.3). The execl call overlays the address space of the child 
process with the file “copy”, assumed to be in the current directory, and runs the 
program with the user-supplied parameters. If the execl call succeeds, it never 
returns because the process executes in a new address space, as will be seen in 
Chapter 7. Meanwhile, the process that had invoked fork (the parent) receives a 
non-0 return from the call, calls wait , suspending its execution until copy finishes, 
prints the message “copy done,” and exits (every program exits at the end of its 
main function, as arranged by standard C program libraries that are linked during 
the compilation process). For example, if the name of the executable program is 
run , and a user invokes the program by 
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main(argc, argv) 
int argc; 
char *argv[]; 

{ 

/* assume 2 args: source file and target file */ 
if (forkO *“ 0) 

cxeclC’copy", "copy", argv[l], argv[2], 0); 
wait((int *) 0); 
printf("copy done\n"); 

} 



Figure 1.4. Program that Creates a New Process to Copy Files 



run oldfile newfile 

the process copies “oldfile” to “newfile” and prints out the message. Although this 
program adds little to the “copy” program, it exhibits four major system calls used 
for process control: fork , exec, wait , and, discreetly, exit. 

Generally, the system calls allow users to write programs that do sophisticated 
operations, and as a result, the kernel of the UNIX system does not contain many 
functions that are part of the “kernel” in other systems. Such functions, including 
compilers and editors, are user-level programs in the UNIX system. The prime 
example of such a program is the shelf the command interpreter program that 
users typically execute after logging into the system. The shell interprets the first 
word of a command line as a command name: for many commands, the shell fork s 
and the child process execs the command associated with the name, treating the 
remaining words on the command line as parameters to the command. 

The shell allows three types of commands. First, a command can be an 
executable file that contains object code produced by compilation of source code (a 
C program for example). Second, a command can be an executable file that 
contains a sequence of shell command lines. Finally, a command can be an internal 
shell command (instead of an executable file). The internal commands make the 
shell a programming language in addition to a command interpreter and include 
commands for looping (for -in -do -done and while -do- done), commands for 
conditional execution (if-then-else-fi), a “case” statement command, a command to 
change the current directory of a process ( cd ), and several others. The shell syntax 
allows for pattern matching and parameter processing. Users execute commands 
without having to know their types. 

The shell searches for commands in a given sequence of directories, changeable 
by user request per invocation of the shell. The shell usually executes a command 
synchronously, waiting for the command to terminate before reading the next 
command line. However, it also allows asynchronous execution, where it reads the 
next command line and executes it without waiting for the prior command to 
terminate. Commands executed asynchronously are said to execute in the 
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background. For example, typing the command 

who 

causes the system to execute the program stored in the file fbin/who , 4 which prints a 
list of people who are currently logged in to the system. While who executes, the 
shell waits for it to finish and then prompts the user for another command. By 
typing 

who & 

the system executes the program who in the background, and the shell is ready to 
accept another command immediately. 

Every process executing in the UNIX system has an execution environment that 
includes a current directory. The current directory of a process is the start 
directory used for all path names that do not begin with the slash character. The 
user may execute the shell command cd, change directory, to move around the file 
system tree and change the current directory. The command line 

cd /usr/sre/uts 

changes the shell’s current directory to the directory “/usr/sre/uts”. The command 
line 

cd 

changes the shell’s current directory to the directory that is two nodes “closer” to 
the root node: the component refers to the parent directory of the current 
directory. 

Because the shell is a user program and not part of the kernel, it is easy to 
modify it and tailor it to a particular environment. For instance, users can use the 
C shell to provide a history mechanism and avoid retyping recently used commands, 
instead of the Bourne shell (named after its inventor, Steve Bourne), provided as 
part of the standard System V release. Or some users may be granted use only of 
a restricted shell, providing a scaled down version of the regular shell. The system 
can execute the various shells simultaneously. Users have the capability to execute 
many processes simultaneously, and processes can create other processes 
dynamically and synchronize their execution, if desired. These features provide 
users with a powerful execution environment. Although much of the power of the 
shell derives from its capabilities as a programming language and from its 
capabilities for pattern matching of arguments, this section concentrates on the 
process environment provided by the system via the shell. Other important shell 



4. The directory “/bin” contains many useful commands and is usually included in the sequence of 
directories the shell searches. 
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features are beyond the scope of this book (see [Bourne 78] for a detailed 
description of the shell). 



1.3.3 Building Block Primitives 

As described earlier, the philosophy of the UNIX system is to provide operating 
system primitives that enable users to write small, modular programs that can be 
used as building blocks to build more complex programs. One such primitive 
visible to shell users is the capability to redirect I/O. Processes conventionally have 
access to three files: they read from their standard input file, write to their 

standard output file, and write error messages to their standard error file. 
Processes executing at a terminal typically use the terminal for these three files, but 
each may be “redirected” independently. For instance, the command line 

Is 

lists all files in the current directory on the standard output, but the command line 
Is > output 

redirects the standard output to the file called “output” in the current directory, 
using the creat system call mentioned above. Similarly, the command line 

mail mjb < letter 

opens the file “letter” for its standard intput and mails its contents to the user 
named “mjb.” Processes can redirect input and output simultaneously, as in 

nroff —mm < docl > doc 1. out 2> errors 

where the text formatter nroff reads the input file docl , redirects its standard 
output to the file docl. out , and redirects error messages to the file errors (the 
notation “2>” means to redirect the output for file descriptor 2, conventionally the 
standard error). The programs Is , mail, and nroff do not know what file their 
standard input, standard output, or standard error will be; the shell recognizes the 
symbols “<”, and “2>” and sets up the standard input, standard output, 

and standard error appropriately before executing the processes. 

The second building block primitive is the pipe, a mechanism that allows a 
stream of data to be passed between reader and writer processes. Processes can 
redirect their standard output to a pipe to be read by other processes that have 
redirected their standard input to come from the pipe. The data that the first 
processes write into the pipe is the input for the second processes. The second 
processes could also redirect their output, and so on, depending on programming 
need. Again, the processes need not know what type of file their standard output is; 
they work regardless of whether their standard output is a regular file, a pipe, or a 
device. When using the smaller programs as building blocks for a larger, more 
complex program, the programmer uses the pipe primitive and redirection of I/O to 
integrate the piece parts. Indeed, the system tacitly encourages such programming 
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style so that new programs can work with existing programs. 

For example, the program grep searches a set of files (parameters to grcp) for a 
given pattern: 

grep main a.c b.c c.c 

searches the three files a.c, b.c, and c.c for lines containing the string “main” and 
prints the lines that it finds onto standard output. Sample output may be: 

a.c: main(argc, argv) 

c.c: /* here is the main loop in the program */ 
c.c: mainO 

The program wc with the option -1 counts the number of lines in the standard 
input file. The command line 

grep main a.c b.c c.c | wc —1 

counts the number of lines in the files that contain the string “main”; the output 
from grep is “piped” directly into the wc command. For the previous sample 
output from grep y the output from the piped command is 

3 

The use of pipes frequently makes it unnecessary to create temporary files. 



1.4 OPERATING SYSTEM SERVICES 

Figure 1.1 depicts the kernel layer immediately below the layer of user application 
programs. The kernel performs various primitive operations on behalf of user 
processes to support the user interface described above. Among the services 
provided by the kernel are 

• Controlling the execution of processes by allowing their creation, termination or 
suspension, and communication 

• Scheduling processes fairly for execution on the CPU. Processes share the CPU 
in a time-shared manner: the CPU 5 executes a process, the kernel suspends it 
when its time quantum elapses, and the kernel schedules another process to 
execute. The kernel later reschedules the suspended process. 

• Allocating main memory for an executing process. The kernel allows processes 
to share portions of their address space under certain conditions, but protects 
the private address space of a process from outside tampering. If the system 
runs low on free memory, the kernel frees memory by writing a process 



5. Chapter 12 will consider multiprocessor systems; until then, assume a single processor model. 
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temporarily to secondary memory, called a swap device. If the kernel writes 
entire processes to a swap device, the implementation of the UNIX system is 
called a swapping system; if it writes pages of memory to a swap device, it is 
called a paging system. 

• Allocating secondary memory for efficient storage and retrieval of user data. 
This service constitutes the file system. The kernel allocates secondary storage 
for user files, reclaims unused storage, structures the file system in a well 
understood manner, and protects user files from illegal access. 

• Allowing processes controlled access to peripheral devices such as terminals, 
tape drives, disk drives, and network devices. 

The kernel provides its services transparently. For example, it recognizes that a 
given file is a regular file or a device, but hides the distinction from user processes. 
Similarly, it formats data in a file for internal storage, but hides the internal format 
from user processes, returning an unformatted byte stream. Finally, it offers 
necessary services so that user-level processes can support the services they must 
provide, while omitting services that can be implemented at the user level. For 
example, the kernel supports the services that the shell needs to act as a command 
interpreter: It allows the shell to read terminal input, to spawn processes 

dynamically, to synchronize process execution, to create pipes, and to redirect I/O. 
Users can construct private versions of the shell to tailor their environments to their 
specifications without affecting other users. These programs use the same kernel 
services as the standard shell. 



1.5 ASSUMPTIONS ABOUT HARDWARE 

The execution of user processes on UNIX systems is divided into two levels: user 
and kernel. When a process executes a system call, the execution mode of the 
process changes from user mode to kernel mode : the operating system executes 
and attempts to service the user request, returning an error code if it fails. Even if 
the user makes no explicit requests for operating system services, the operating 
system still does bookkeeping operations that relate to the user process, handling 
interrupts, scheduling processes, managing memory, and so on. Many machine 
architectures (and their operating systems) support more levels than the two 
outlined here, but the two modes, user and kernel, are sufficient for UNIX systems. 
The differences between the two modes are 

• Processes in user mode can access their own instructions and data but not kernel 
instructions and data (or those of other processes). Processes in kernel mode, 
however, can access kernel and user addresses. For example, the virtual address 
space of a process may be divided between addresses that are accessible only in 
kernel mode and addresses that are accessible in either mode. 

• Some machine instructions are privileged and result in an error when executed 
in user mode. For example, a machine may contain an instruction that 
manipulates the processor status register; processes executing in user mode 
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Figure 1.5. Multiple Processes and Modes of Execution 



should not have this capability. 

Put simply, the hardware views the world in terms of kernel mode and user mode 
and does not distinguish among the many users executing programs in those modes. 
The operating system keeps internal records to distinguish the many processes 
executing on the system. Figure 1.5 shows the distinction: the kernel distinguishes 
between processes A, B, C, and D on the horizontal axis, and the hardware 
distinguishes the mode of execution on the vertical axis. 

Although the system executes in one of two modes, the kernel runs on behalf of 
a user process. The kernel is not a separate set of processes that run in parallel to 
user processes, but it is part of each user process. The ensuing text will frequently 
refer to “the kernel” allocating resources or “the kernel” doing various operations, 
but what is meant is that a process executing in kernel mode allocates the resources 
or does the various operations. For example, the shell reads user terminal input via 
a system call: The kernel, executing on behalf of the shell process, controls the 
operation of the terminal and returns the typed characters to the shell. The shell 
then executes in user mode, interprets the character stream typed by the user, and 
does the specified set of actions, which may require invocation of other system calls. 



1.5.1 Interrupts and Exceptions 

The UNIX system allows devices such as I/O peripherals or the system clock to 
interrupt the CPU asynchronously. On receipt of the interrupt, the kernel saves its 
current context (a frozen image of what the process was doing), determines the 
cause of the interrupt, and services the interrupt. After the kernel services the 
interrupt, it restores its interrupted context and proceeds as if nothing had 
happened. The hardware usually prioritizes devices according to the order that 
interrupts should be handled: When the kernel services an interrupt, it blocks out 
lower priority interrupts but services higher priority interrupts. 

An exception condition refers to unexpected events caused by a process, such as 
addressing illegal memory, executing privileged instructions, dividing by zero, and 
so on. They are distinct from interrupts, which are caused by events that are 
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external to a process. Exceptions happen “in the middle” of the execution of an 
instruction, and the system attempts to restart the instruction after handling the 
exception; interrupts are considered to happen between the execution of two 
instructions, and the system continues with the next instruction after servicing the 
interrupt. The UNIX system uses one mechanism to handle interrupts and 
exception conditions. 



1.5.2 Processor Execution Levels 

The kernel must sometimes prevent the occurrence of interrupts during critical 
activity, which could result in corrupt data if interrupts were allowed. For instance, 
the kernel may not want to receive a disk interrupt while manipulating linked lists, 
because handling the interrupt could corrupt the pointers, as will be seen in the 
next chapter. Computers typically have a set of privileged instructions that set the 
processor execution level in the processor status word. Setting the processor 
execution level to certain values masks off interrupts from that level and lower 
levels, allowing only higher-level interrupts. Figure 1.6 shows a sample set of 
execution levels. If the kernel masks out disk interrupts, all interrupts except for 
clock interrupts and machine error interrupts are prevented. If it masks out 
software interrupts, all other interrupts may occur. 
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Figure 1.6. Typical Interrupt Levels 



1.5.3 Memory Management 

The kernel permanently resides in main memory as does the currently executing 
process (or parts of it, at least). When compiling a program, the compiler 
generates a set of addresses in the program that represent addresses of variables 





18 



GENERAL OVERVIEW OF THE SYSTEM 



and data structures or the addresses of instructions such as functions. The compiler 
generates the addresses for a virtual machine as if no other program will execute 
simultaneously on the physical machine. 

When the program is to run on the machine, the kernel allocates space in main 
memory for it, but the virtual addresses generated by the compiler need not be 
identical to the physical addresses that they occupy in the machine. The kernel 
coordinates with the machine hardware to set up a virtual to physical address 
translation that maps the compiler-generated addresses to the physical machine 
addresses. The mapping depends on the capabilities of the machine hardware, and 
the parts of UNIX systems that deal with them are therefore machine dependent. 
For example, some machines have special hardware to support demand paging. 
Chapters 6 and 9 will discuss issues of memory management and how they relate to 
hardware in more detail. 



1.6 SUMMARY 

This chapter has described the overall structure of the UNIX system, the 
relationship between processes running in user mode versus kernel mode, and the 
assumptions the kernel makes about the hardware. Processes execute in user mode 
or kernel mode, where they avail themselves of system services using a well-defined 
set of system calls. The system design encourages programmers to write small 
programs that do only a few operations but do them well, and then to combine the 
programs using pipe s and I/O redirection to do more sophisticated processing. 

The system calls allow processes to do operations that are otherwise forbidden to 
them. In addition to servicing system calls, the kernel does general bookkeeping for 
the user community, controlling process scheduling, managing the storage and 
protection of processes in main memory, fielding interrupts, managing files and 
devices, and taking care of system error conditions. The UNIX system kernel 
purposely omits many functions that are part of other operating systems, providing 
a small set of system calls that allow processes to do necessary functions at user 
level. The next chapter gives a more detailed introduction to the kernel, describing 
its architecture and some basic concepts used in its implementation. 
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The last chapter gave a high-level perspective of the UNIX system environment. 
This chapter focuses on the kernel, providing an overview of its architecture and 
outlining basic concepts and structures essential for understanding the rest of the 
book. 



2.1 ARCHITECTURE OF THE UNIX OPERATING SYSTEM 

It has been noted (see page 239 of [Christian 83]) that the UNIX system supports 
the illusions that the file system has “places” and that processes have “life.” The 
two entities, files and processes, are the two central concepts in the UNIX system 
model. Figure 2.1 gives a block diagram of the kernel, showing various modules 
and their relationships to each other. In particular, it shows the file subsystem on 
the left and the process control subsystem on the right, the two major components 
of the kernel. The diagram serves as a useful logical view of the kernel, although 
in practice the kernel deviates from the model because some modules interact with 
the internal operations of others. 

Figure 2.1 shows three levels: user, kernel, and hardware. The system call and 
library interface represent the border between user programs and the kernel 
depicted in Figure 1.1. System calls look like ordinary function calls in C 
programs, and libraries map these function calls to the primitives needed to enter 
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Figure 2.1. Block Diagram of the System Kernel 



the operating system, as covered in more detail in Chapter 6. Assembly language 
programs may invoke system calls directly without a system call library, however. 
Programs frequently use other libraries such as the standard I/O library to provide 
a more sophisticated use of the system calls. The libraries are linked with the 
programs at compile time and are thus part of the user program for purposes of 
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this discussion. An example later on will illustrate these points. 

The figure partitions the set of system calls into those that interact with the file 
subsystem and those that interact with the process control subsystem. The file 
subsystem manages files, allocating file space, administering free space, controlling 
access to files, and retrieving data for users. Processes interact with the file 
subsystem via a specific set of system calls, such as open (to open a file for reading 
or writing), close , read , write , stat (query the attributes of a file), chown (change 
the record of who owns the file), and chmod (change the access permissions of a 
file). These and others will be examined in Chapter 5. 

The file subsystem accesses file data using a buffering mechanism that regulates 
data flow between the kernel and secondary storage devices. The buffering 
mechanism interacts with block I/O device drivers to initiate data transfer to and 
from the kernel. Device drivers are the kernel modules that control the operation 
of peripheral devices. Block I/O devices are random access storage devices; 
alternatively, their device drivers make them appear to be random access storage 
devices to the rest of the system. For example, a tape driver may allow the kernel 
to treat a tape unit as a random access storage device. The file subsystem also 
interacts directly with “raw” I/O device drivers without the intervention of a 
buffering mechanism. Raw devices, sometimes called character devices, include all 
devices that are not block devices. 

The process control subsystem is responsible for process synchronization, 
interprocess communication, memory management, and process scheduling. The 
file subsystem and the process control subsystem interact when loading a file into 
memory for execution, as will be seen in Chapter 7: the process subsystem reads 
executable files into memory before executing them. 

Some of the system calls for controlling processes are fork (create a new 
process), exec (overlay the image of a program onto the running process), exit 
(finish executing a process), wait (synchronize process execution with the exit of a 
previously forked process), brk (control the size of memory allocated to a process), 
and signal (control process response to extraordinary events). Chapter 7 will 
examine these system calls and others. 

The memory management module controls the allocation of memory. If at any 
time the system does not have enough physical memory for all processes, the kernel 
moves them between main memory and secondary memory so that all processes get 
a fair chance to execute. Chapter 9 will describe two policies for managing 
memory: swapping and demand paging. The swapper process is sometimes called 
the scheduler, because it “schedules” the allocation of memory for processes and 
influences the operation of the CPU scheduler. However, this text will refer to it as 
the swapper to avoid confusion with the CPU scheduler. 

The scheduler module allocates the CPU to processes. It schedules them to run 
in turn until they voluntarily relinquish the CPU while awaiting a resource or until 
the kernel preempts them when their recent run time exceeds a time quantum. The 
scheduler then chooses the highest priority eligible process to run; the original 
process will run again when it is the highest priority eligible process available. 
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There are several forms of interprocess communication, ranging from asynchronous 
signaling of events to synchronous transmission of messages between processes. 

Finally, the hardware control is responsible for handling interrupts and for 
communicating with the machine. Devices such as disks or terminals may interrupt 
the CPU while a process is executing. If so, the kernel may resume execution of 
the interrupted process after servicing the interrupt: Interrupts are not serviced by 
special processes but by special functions in the kernel, called in the context of the 
currently running process. 



2.2 INTRODUCTION TO SYSTEM CONCEPTS 

This section gives an overview of some major kernel data structures and describes 
the function of modules shown in Figure 2.1 in more detail. 



2.2.1 An Overview of the File Subsystem 

The internal representation of a file is given by an inode , which contains a 
description of the disk layout of the file data and other information such as the file 
owner, access permissions, and access times. The term inode is a contraction of the 
term index node and is commonly used in literature on the UNIX system. Every 
file has one inode, but it may have several names, all of which map into the inode. 
Each name is called a link. When a process refers to a file by name, the kernel 
parses the file name one component at a time, checks that the process has 
permission to search the directories in the path, and eventually retrieves the inode 
for the file. For example, if a process calls 

open C7fs2/mjb/rje/sourcefile’\ 1 ) ; 

the kernel retrieves the inode for “/fs2/mjb/rje/sourcefile”. When a process 
creates a new file, the kernel assigns it an unused inode. Inodes are stored in the 
file system, as will be seen shortly, but the kernel reads them into an in-core 1 inode 
table when manipulating files. 

The kernel contains two other data structures, the file table and the user file 
descriptor table. The file table is a global kernel structure, but the user file 
descriptor table is allocated per process. When a process opens or creat s a file, the 
kernel allocates an entry from each table, corresponding to the file’s inode. Entries 
in the three structures — user file descriptor table, file table, and inode table — 
maintain the state of the file and the user’s access to it. The file table keeps track 
of the byte offset in the file where the user’s next read or write will start, and the 



1. The term core refers to primary memory of a machine, not to hardware technology. 




2.2 



INTRODUCTION TO SYSTEM CONCEPTS 



23 



User 

File Descriptor File Inode 

Table Table Table 



X 










- 










X 


1 

1 

1 

it 

1/ 







> 




X 










X 










X 










X 










X 

> 


^ 1 




> 



Figure 2.2. File Descriptors, File Table, and Inode Table 



access rights allowed to the opening process. The user file descriptor table 
identifies all open files for a process. Figure 2.2 shows the tables and their 
relationship to each other. The kernel returns a file descriptor for the open and 
creat system calls, which is an index into the user file descriptor table. When 
executing read and write system calls, the kernel uses the file descriptor to access 
the user file descriptor table, follows pointers to the file table and inode table 
entries, and, from the inode, finds the data in the file. Chapters 4 and 5 describe 
these data structures in great detail. For now, suffice it to say that use of three 
tables allows various degrees of sharing access to a file. 

The UNIX system keeps regular files and directories on block devices such as 
tapes or disks. Because of the difference in access time between the two, few, if 
any, UNIX system installations use tapes for their file systems. In coming years, 
diskless work stations will be common, where files are located on a remote system 
and accessed via a network (see Chapter 13). For simplicity, however, the ensuing 
text assumes the use of disks. An installation may have several physical disk units, 
each containing one or more file systems. Partitioning a disk into several file 
systems makes it easier for administrators to manage the data stored there. The 
kernel deals on a logical level with file systems rather than with disks, treating each 
one as a logical device identified by a logical device number. The conversion 
between logical device (file system) addresses and physical device (disk) addresses 
is done by the disk driver. This book will use the term device to mean a logical 
device unless explicitly stated otherwise. 

A file system consists of a sequence of logical blocks, each containing 512, 1024, 
2048, or any convenient multiple of 512 bytes, depending on the system 
implementation. The size of a logical block is homogeneous within a file system but 
may vary between different file systems in a system configuration. Using large 
logical blocks increases the effective data transfer rate between disk and memory, 
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because the kernel can transfer more data per disk operation and therefore make 
fewer time-consuming operations. For example, reading IK bytes from a disk in 
one read operation is faster than reading 512 bytes twice. However, if a logical 
block is too large, effective storage capacity may drop, as will be shown in Chapter 
5. For simplicity, this book will use the term “block” to mean a logical block, and 
it will assume that a logical block contains IK bytes of data unless explicitly stated 
otherwise. 



boot 

block 



super 

block 



inode list data blocks 

Figure 2.3. File System Layout 



A file system has the following structure (Figure 2.3). 

• The boot block occupies the beginning of a file system, typically the first sector, 
and may contain the bootstrap code that is read into the machine to boot , or 
initialize, the operating system. Although only one bool block is needed to boot 
the system, every file system has a (possibly empty) boot block. 

• The super block describes the state of a file system — how large it is, how 
many files it can store, where to find free space on the file system, and other 
information. 

• The inode list is a list of inodes that follows the super block in the file system. 
Administrators specify the size of the inode list when configuring a file system. 
The kernel references inodes by index into the inode list. One inode is the root 
inode of the file system: it is the inode by which the directory structure of the 
file system is accessible after execution of the mount system call (Section 5.14). 

• The data blocks start at the end of the inode list and contain file data and 
administrative data. An allocated data block can belong to one and only one 
file in the file system. 



2.2.2 Processes 

This section examines the process subsystem more closely. It describes the 
structure of a process and some process data structures used for memory 
management. Then it gives a preliminary view of the process state diagram and 
considers various issues involved in some state transitions. 

A process is the execution of a program and consists of a pattern of bytes that 
the CPU interprets as machine instructions (called “text”), data, and stack. Many 
processes appear to execute simultaneously as the kernel schedules them for 
execution, and several processes may be instances of one program. A process 
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executes by following a strict sequence of instructions that is self-contained and 
does not jump to that of another process; it reads and writes its data and stack 
sections, but it cannot read or write the data and stack of other processes. 
Processes communicate with other processes and with the rest of the world via 
system calls. 

In practical terms, a process on a UNIX system is the entity that is created by 
the fork system call. Every process except process 0 is created when another 
process executes the fork system call. The process that invoked the fork system 
call is the parent process, and the newly created process is the child process. Every 
process has one parent process, but a process can have many child processes. The 
kernel identifies each process by its process number, called the process ID (PID). 
Process 0 is a special process that is created “by hand” when the system boots; 
after fork ing a child process (process 1), process 0 becomes the swapper process. 
Process 1, known as init, is the ancestor of every other process in the system and 
enjoys a special relationship with them, as explained in Chapter 7. 

A user compiles the source code of a program to create an executable file, which 
consists of several parts: 

• a set of “headers” that describe the attributes of the file, 

• the program text, 

• a machine language representation of data that has initial values when the 
program starts execution, and an indication of how much space the kernel 
should allocate for uninitialized data, called bss 2 (the kernel initializes it to 0 at 
run time), 

• other sections, such as symbol table information. 

For the program in Figure 1.3, the text of the executable file is the generated code 
for the functions main and copy , the initialized data is the variable version (put 
into the program just so that it should have some initialized data), and the 
uninitialized data is the array buffer. System V versions of the C compiler create a 
separate text section by default but support an option that allows inclusion of 
program instructions in the data section, used in older versions of the system. 

The kernel loads an executable file into memory during an exec system call, and 
the loaded process consists of at least three parts, called regions : text, data, and 
the stack. The text and data regions correspond to the text and data-bss sections of 
the executable file, but the stack region is automatically created and its size is 
dynamically adjusted by the kernel at run time. The stack consists of logical stack 
frames that are pushed when calling a function and popped when returning; a 
special register called the stack pointer indicates the current stack depth. A stack 



2. The name bss comes from an assembly pseudo-operator on the IBM 7090 machine, which stood for 
“block started by symbol.” 
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frame contains the parameters to a function, its local variables, and the data 
necessary to recover the previous stack frame, including the value of the program 
counter and stack pointer at the time of the function call. The program code 
contains instruction sequences that manage stack growth, and the kernel allocates 
space for the stack, as needed. In the program in Figure 1.3, parameters argc and 
argv and variables fdold and fdnew in the function main appear on the stack when 
main is called (once in every program, by convention), and parameters old and new 
and the variable count in the function copy appear on the stack whenever copy is 
called. 

Because a process in the UNIX system can execute in two modes, kernel or 
user, it uses a separate stack for each mode. The user stack contains the 
arguments, local variables, and other data for functions executing in user mode. 
The left side of Figure 2.4 shows the user stack for a process when it makes the 
write system call in the copy program. The process startup procedure (included in 
a library) had called the function main with two parameters, pushing frame 1 onto 
the user stack; frame 1 contains space for the two local variables of main. Main 
then called copy with two parameters, old and new , and pushed frame 2 onto the 
user stack; frame 2 contains space for the local variable count . Finally, the process 
invoked the system call write by invoking the library function write. Each system 
call has an entry point in a system call library; the system call library is encoded in 
assembly language and contains special trap instructions, which, when executed, 
cause an “interrupt” that results in a hardware switch to kernel mode. A process 
calls the library entry point for a particular system call just as it calls any function, 
creating a stack frame for the library function. When the process executes the 
special instruction, it switches mode to the kernel, executes kernel code, and uses 
the kernel stack. 

The kernel stack contains the stack frames for functions executing in kernel 
mode. The function and data entries on the kernel stack refer to functions and 
data in the kernel, not the user program, but its construction is the same as that of 
the user stack. The kernel stack of a process is null when the process executes in 
user mode. The right side of Figure 2.4 depicts the kernel stack representation for 
a process executing the write system call in the copy program. The names of the 
algorithms are described during the detailed discussion of the write system call in 
later chapters. 

Every process has an entry in the kernel process table , and each process is 
allocated a u area 3 that contains private data manipulated only by the kernel. The 
process table contains (or points to) a per process region table , whose entries point 
to entries in a region table. A region is a contiguous area of a process’s address 



3. The u in u area stands for “user.” Another name for the u area is u block; this book will always 
refer to it as the u area. 
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per process 




space* such as text, data, and stack. Region table entries describe the attributes of 
the region, such as whether it contains text or data, whether it is shared or private, 
and where the “data” of the region is located in memory. The extra level of 
indirection (from the per process region table to the region table) allows 
independent processes to share regions. When a process invokes the exec system 
call, the kernel allocates regions for its text, data, and stack after freeing the old 
regions the process had been using. When a process invokes fork , the kernel 
duplicates the address space of the old process, allowing processes to share regions 
when possible and making a physical copy otherwise. When a process invokes exit , 
the kernel frees the regions the process had used. Figure 2.5 shows the relevant 
data structures of a running process: The process table points to a per process 
region table with pointers to the region table entries for the text, data, and stack 
regions of the process. 

The process table entry and the u area contain control and status information 
about the process. The u area is an extension of the process table entry, and 
Chapter 6 will examine the distinction between the two tables. Fields in the 
process table discussed in the following chapters are 

• a state field, 

• identifiers indicating the user who owns the process (user IDs, or UIDs), 

• an event descriptor set when a process is suspended (in the sleep state). 

The u area contains information describing the process that needs to be 
accessible only when the process is executing. The important fields are 
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• a pointer to the process table slot of the currently executing process, 

• parameters of the current system call, return values and error codes, 

• file descriptors for all open files, 

• internal I/O parameters, 

• current directory and current root (see Chapter 5), 

• process and file size limits. 

The kernel can directly access fields of the u area of the executing process but not 
of the u area of other processes. Internally, the kernel references the structure 
variable u to access the u area of the currently running process, and when another 
process executes, the kernel rearranges its virtual address space so that the 
structure u refers to the u area of the new process. The implementation gives the 
kernel an easy way to identify the current process by following the pointer from the 
u area to its process table entry. 



2.2.2. 1 Context of a process 

The context of a process is its state, as defined by its text, the values of its global 
user variables and data structures, the values of machine registers it uses, the 
values stored in its process table slot and u area , and the contents of its user and 
kernel stacks. The text of the operating system and its global data structures are 
shared by all processes but do not constitute part of the context of a process. 

When executing a process, the system is said to be executing in the context of 
the process. When the kernel decides that it should execute another process, it does 
a context switch , so that the system executes in the context of the other process. 
The kernel allows a context switch only under specific conditions, as will be seen. 
When doing a context switch, the kernel saves enough information so that it can 
later switch back to the first process and resume its execution. Similarly, when 
moving from user to kernel mode, the kernel saves enough information so that it 
can later return to user mode and continue execution from where it left off. 
Moving between user and kernel mode is a change in mode, not a context switch. 
Recalling Figure 1.5, the kernel does a context switch when it changes context from 
process A to process B; it changes execution mode from user to kernel or from 
kernel to user, still executing in the context of one process, such as process A. 

The kernel services interrupts in the context of the interrupted process even 
though it may not have caused the interrupt. The interrupted process may have 
been executing in user mode or in kernel mode. The kernel saves enough 
information so that it can later resume execution of the interrupted process and 
services the interrupt in kernel mode. The kernel does not spawn or schedule a 
special process to handle interrupts. 
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2.2.2.2 Process states 

The lifetime of a process can be divided into a set of states , each with certain 
characteristics that describe the process. Chapter 6 will describe all process states, 
but it is essential to understand the following states now: 

1 . The process is currently executing in user mode. 

2. The process is currently executing in kernel mode. 

3. The process is not executing, but it is ready to run as soon as the scheduler 
chooses it. Many processes may be in this state, and the scheduling 
algorithm determines which one will execute next. 

4. The process is sleeping . A process puts itself to sleep when it can no longer 
continue executing, such as when it is waiting for I/O to complete. 

Because a processor can execute only one process at a time, at most one process 
may be in states 1 and 2. The two states correspond to the two modes of execution, 
user and kernel. 



2.2.2.3 State transitions 

The process states described above give a static view of a process, but processes 
move continuously between the states according to well-defined rules. A state 
transition diagram is a directed graph whose nodes represent the states a process 
can enter and whose edges represent the events that cause a process to move from 
one state to another. State transitions are legal between two states if there exists 
an edge from the first state to the second. Several transitions may emanate from a 
state, but a process will follow one and only one transition depending on the system 
event that occurs. Figure 2.6 shows the state transition diagram for the process 
states defined above. 

Several processes can execute simultaneously in a time-shared manner, as stated 
earlier, and they may all run simultaneously in kernel mode. If they were allowed 
to run in kernel mode without constraint, they could corrupt global kernel data 
structures. By prohibiting arbitrary context switches and controlling the occurrence 
of interrupts, the kernel protects its consistency. 

The kernel allows a context switch only when a process moves from the state 
“kernel running” to the state “asleep in memory.” Processes running in kernel 
mode cannot be preempted by other processes; therefore the kernel is sometimes 
said to be non- preemptive , although the system does preempt processes that are in 
user mode. The kernel maintains consistency of its data structures because it is 
non-preemptive, thereby solving the mutual exclusion problem — making sure that 
critical sections of code are executed by at most one process at a time. 

For instance, consider the sample code in Figure 2.7 to put a data structure, 
whose address is in the pointer bpl, onto a doubly linked list after the structure 
whose address is in bp. If the system allowed a context switch while the kernel 
executed the code fragment, the following situation could occur. Suppose the 
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Figure 2.6. Process States and Transitions 



kernel executes the code until the comment and then does a context switch. The 
doubly linked list is in an inconsistent state: the structure bpl is half on and half 
off the linked list. If a process were to follow the forward pointers, it would find 
bpl on the linked list, but if it were to follow the back pointers, it would not find 
bpl (Figure 2.8). If other processes were to manipulate the pointers on the linked 
list before the original process ran again, the structure of the doubly linked list 
could be permanently destroyed. The UNIX system prevents such situations by 
disallowing context switches when a process executes in kernel mode. If a process 
goes to sleep, thereby permitting a context switch, kernel algorithms are encoded to 
make sure that system data structures are in a safe, consistent state. 

A related problem that can cause inconsistency in kernel data is the handling of 
interrupts, which can change kernel state information. For example, if the kernel 
was executing the code in Figure 2.7 and received an interrupt when it reached the 
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struct queue ( 



) *bp, *bpl; 

bpl— >forp — bp— >forp; 
bpl— >backp — bp; 
bp->forp - bpl; 

/* consider possible context switch here V 
bp 1 — > forp— > backp — bpl; 



Figure 2.7. Sample Code Creating Doubly Linked List 




Placing bpl on doubly linked list 




Figure 2.8. Incorrect Linked List because of Context Switch 



comment, the interrupt handler could corrupt the links if it manipulates the 
pointers, as illustrated earlier. To solve this problem, the system could prevent all 
interrupts while executing in kernel mode, but that would delay servicing of the 
interrupt, possibly hurting system throughput. Instead, the kernel raises the 
processor execution level to prevent interrupts when entering critical regions of 
code. A section of code is critical if execution of arbitrary interrupt handlers could 
result in consistency problems. For example, if a disk interrupt handler 
manipulates the buffer queues in the figure, the section of code where the kernel 
manipulates the buffer queues is a critical region of code with respect to the disk 
interrupt handler. Critical regions are small and infrequent so that system 
throughput is largely unaffected by their existence. Other operating systems solve 
this problem by preventing all interrupts when executing in system states or by 
using elaborate locking schemes to ensure consistency. Chapter 12 will return to 
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this issue for multiprocessor systems, where the solution outlined here is insufficient. 

To review, the kernel protects its consistency by allowing a context switch only 
when a process puts itself to sleep and by preventing one process from changing the 
state of another process. It also raises the processor execution level around critical 
regions of code to prevent interrupts that could otherwise cause inconsistencies. 
The process scheduler periodically preempts processes executing in user mode so 
that processes cannot monopolize use of the CPU. 



2.2.2.4 Sleep and wakeup 

A process executing in kernel mode has great autonomy in deciding what it is going 
to do in reaction to system events. Processes can communicate with each other and 
“suggest” various alternatives, but they make the final decision by themselves. As 
will be seen, there is a set of rules that processes obey when confronted with various 
circumstances, but each process ultimately follows these rules under its own 
initiative. For instance, when a process must temporarily suspend its execution 
(“go to sleep”), it does so of its own free will. Consequently, an interrupt handler 
cannot go to sleep, because if it could, the interrupted process would be put to sleep 
by default. 

Processes go to sleep because they are awaiting the occurrence of some event, 
such as waiting for I/O completion from a peripheral device, waiting for a process 
to exit, waiting for system resources to become available, and so on. Processes are 
said to sleep on an event , meaning that they are in the sleep state until the event 
occurs, at which time they wake up and enter the state “ready to run.” Many 
processes can simultaneously sleep on an event; when an event occurs, all processes 
sleeping on the event wake up because the event condition is no longer true. When 
a process wakes up, it follows the state transition from the “sleep” state to the 
“ready-to-run” state, where it is eligible for later scheduling; it does not execute 
immediately. Sleeping processes do not consume CPU resources: The kernel does 
not constantly check to see that a process is still sleeping but waits for the event to 
occur and awakens the process then. 

For example, a process executing in kernel mode must sometimes lock a data 
structure in case it goes to sleep at a later stage; processes attempting to 
manipulate the locked structure must check the lock and sleep if another process 
owns the lock. The kernel implements such locks in the following manner: 

while (condition is true) 

sleep (event: the condition becomes false); 

set condition true; 

It unlocks the lock and awakens all processes asleep on the lock in the following 



manner: 
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set condition false; 

wakcup (event: the condition is false); 

Figure 2.9 depicts a scenario where three processes, A, B, and C, contend for a 
locked buffer. The sleep condition is that the buffer is locked. The processes 
execute one at a time, find the buffer locked, and sleep on the event that the buffer 
becomes unlocked. Eventually, the buffer is unlocked, and all processes wake up 
and enter the state “ready to run.” The kernel eventually chooses one process, say 
B, to execute. Process B executes the “while” loop, finds that the buffer is 
unlocked, sets the buffer lock, and proceeds. If process B later goes to sleep again 
before unlocking the buffer (waiting for completion of an I/O operation, for 
example), the kernel can schedule other processes to run. If it chooses process A, 
process A executes the “while” loop, finds that the buffer is locked, and goes to 
sleep again; process C may do the same thing. Eventually, process B awakens and 
unlocks the buffer, allowing either process A or C to gain access to the buffer. 
Thus, the “while-sleep” loop insures that at most one process can gain access to a 
resource. 

Chapter 6 will present the algorithms for sleep and wakeup in greater detail. In 
the meantime, they should be considered “atomic”: A process enters the sleep state 
instantaneously and stays there until it wakes up. After it goes to sleep, the kernel 
schedules another process to run and switches context to it. 



2.3 KERNEL DATA STRUCTURES 

Most kernel data structures occupy fixed-size tables rather than dynamically 
allocated space. The advantage of this approach is that the kernel code is simple, 
but it limits the number of entries for a data structure to the number that was 
originally configured when generating the system: If, during operation of the 
system, the kernel should run out of entries for a data structure, it cannot allocate 
space for new entries dynamically but must report an error to the requesting user. 
If, on the other hand, the kernel is configured so that it it is unlikely to run out of 
table space, the extra table space may be wasted because it cannot be* used for 
other purposes. Nevertheless, the simplicity of the kernel algorithms has generally 
been considered more important than the need to squeeze out every last byte of 
main memory. Algorithms typically use simple loops to find free table entries, a 
method that is easier to understand and sometimes more efficient than more 
complicated allocation schemes. 



2.4 SYSTEM ADMINISTRATION 

Administrative processes are loosely classified as those processes that do various 
functions for the general welfare of the user community. Such functions include 
disk formatting, creation of new file systems, repair of damaged file systems, kernel 
debugging, and others. Conceptually, there is no difference between administrative 
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processes and user processes: They use the same set of system calls available to the 
general community. They are distinguished from general user processes only in the 
rights and privileges they are allowed. For example, file permission modes may 
allow administrative processes to manipulate files otherwise off-limits to general 
users. Internally, the kernel distinguishes a special user called the superuser , 
endowing it with special privileges, as will be seen. A user may become a superuser 
by going through a login-password sequence or by executing special programs. 
Other uses of superuser privileges will be studied in later chapters. In short, the 
kernel does not recognize a separate class of administrative processes. 



2.5 SUMMARY AND PREVIEW 

This chapter has described the architecture of the kernel; its two major components 
are the file subsystem and the process subsystem. The file subsystem controls the 
storage and retrieval of data in user files. Files are organized into file systems, 
which are treated as logical devices; a physical device such as a disk can contain 
several logical devices (file systems). Each file system has a super block that 
describes the structure and contents of the file system, and each file in a file system 
is described by an inode that gives the attributes of the file. System calls that 
manipulate files do so via inodes. 

Processes exist in various states and move between them according to well- 
defined transition rules. In particular, processes executing in kernel mode can 
suspend their execution and enter the sleep state, but no process can put another 
process to sleep. The kernel is non-preemptive, meaning that a process executing in 
kernel mode will continue to execute until it enters the sleep state or until it returns 
to execute in user mode. The kernel maintains the consistency of its data 
structures by enforcing the policy of non-preemption and by blocking interrupts 
when executing critical regions of code. 

The remainder of this text describes the subsystems shown in Figure 2.1 and 
their interactions in detail, starting with the file subsystem and continuing with the 
process subsystem. The next chapter covers the buffer cache and describes buffer 
allocation algorithms, used in the algorithms presented in Chapters 4, 5, and 7. 
Chapter 4 examines internal algorithms of the file system, including the 
manipulation of inodes, the structure of files, and the conversion of path names to 
inodes. Chapter 5 explains the system calls that use the algorithms in Chapter 4 to 
access the file system, such as open , close , read , and write. Chapter 6 deals with 
the basic ideas of the context of a process and its address space, and Chapter 7 
covers system calls that deal with process management and use the algorithms in 
Chapter 6. Chapter 8 examines process scheduling, and Chapter 9 discusses 
memory management algorithms. Chapter 10 covers device drivers, postponed to 
this point so that the relationship between the terminal driver and process 
management can be explained. Chapter 1 1 presents several forms of interprocess 
communication. Finally, the last two chapters cover advanced topics, including 
multiprocessor systems and distributed systems. 
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2.6 EXERCISES 

1 . Consider the following sequence of commands: 

grep main a.c b.c c.c > grepout & 
wc —1 < grepout & 
rm grepout & 

The ampersand ("&”) at the end of each command line informs the shell to run the 
command in the background, and it can execute each command line in parallel. Why 
is this not equivalent to the following command line? 

grep main a.c b.c c.c | wc —1 

2. Consider the sample kernel code in Figure 2.7. Suppose a context switch happens 
when the code reaches the comment, and suppose another process removes a buffer 
from the linked list by executing the following code: 

remove (qp) 

struct queue *qp; 

{ 

qp— >forp->backp — qp— >backp; 
qp— >backp->forp — qp— >forp; 
qp— >forp ™ qjr->backp ™ NULL; 

} 

Consider three cases: 

— The process removes the structure bp from the linked list. 

— The process removes the structure bpl from the linked list. 

— The process removes the structure that follows bpl from the linked list. 

Note, the answer book may be affected by this change. 

What is the status of the linked list after the original process completes executing the 
code after the comment? 

3. What should happen if the kernel attempts to awaken all processes sleeping on an 
event, but no processes are asleep on the event at the time of the wakeup? 
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THE BUFFER 
CACHE 



As mentioned in the previous chapter, the kernel maintains files on mass storage 
devices such as disks, and it allows processes to store new information or to recall 
previously stored information. When a process wants to access data from a file, the 
kernel brings the data into main memory where the process can examine it, alter it, 
and request that the data be saved in the file system again. For example, recall the 
copy program in Figure 1.3: The kernel reads the data from the first file into 
memory, and then writes the data into the second file. Just as it must bring file 
data into memory, the kernel must also bring auxiliary data into memory to 
manipulate it. For instance, the super block of a file system describes the free 
space available on the file system, among other things. The kernel reads the super 
block into memory to access its data and writes it back to the file system when it 
wishes to save its data. Similarly, the inode describes the layout of a file. The 
kernel reads an inode into memory when it wants to access data in a file and writes 
the inode back to the file system when it wants to update the file layout. It 
manipulates this auxiliary data without the explicit knowledge or request of running 
processes. 

The kernel could read and write directly to and from the disk for all file system 
accesses, but system response time and throughput would be poor because of the 
slow disk transfer rate. The kernel therefore attempts to minimize the frequency of 
disk access by keeping a pool of internal data buffers, called the buffer cache, 1 
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which contains the data in recently used disk blocks. 

Figure 2.1 showed the position of the buffer cache module in the kernel 
architecture between the file subsystem and (block) device drivers. When reading 
data from the disk, the kernel attempts to read from the buffer cache. If the data 
is already in the cache, the kernel does not have to read from the disk. If the data 
is not in the cache, the kernel reads the data from the disk and caches it, using an 
algorithm that tries to save as much good data in the cache as possible. Similarly, 
data being written to disk is cached so that it will be there if the kernel later tries 
to read it. The kernel also attempts to minimize the frequency of disk write 
operations by determining whether the data must really be stored on disk or 
whether it is transient data that will soon be overwritten. Higher-level kernel 
algorithms instruct the buffer cache module to pre-cache data or to delay-write 
data to maximize the caching effect. This chapter describes the algorithms the 
kernel uses to manipulate buffers in the buffer cache. 



3.1 BUFFER HEADERS 

During system initialization, the kernel allocates space for a number of buffers, 
configurable according to memory size and system performance constraints. A 
buffer consists of two parts: a memory array that contains data from the disk and 
a buffer header that identifies the buffer. Because there is a one to one mapping of 
buffer headers to data arrays, the ensuing text will frequently refer to both parts as 
a “buffer,” and the context should make clear which part is being discussed. 

The data in a buffer corresponds to the data in a logical disk block on a file 
system, and the kernel identifies the buffer contents by examining identifier fields in 
the buffer header. The buffer is the in-memory copy of the disk block; the contents 
of the disk block map into the buffer, but the mapping is temporary until the kernel 
decides to map another disk block into the buffer. A disk block can never map into 
more than one buffer at a time. If two buffers were to contain data for one disk 
block, the kernel would not know which buffer contained the current data and could 
write incorrect data back to disk. For example, suppose a disk block maps into two 
buffers, A and B. If the kernel writes data first into buffer A and then into buffer 
B, the disk block should contain the contents of buffer B if all write operations 
completely fill the buffer. However, if the kernel reverses the order when it copies 
the buffers to disk, the disk block will contain incorrect data. 

The buffer header (Figure 3.1) contains a device number field and a block 
number field that specify the file system and block number of the data on disk and 
uniquely identify the buffer. The device number is the logical file system number 



1. The buffer cache is a software structure that should not be confused with hardware caches that speed 
memory references. 
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Figure 3.1. Buffer Header 



(see Section 2.2.1), not a physical device (disk) unit number. The buffer header 
also contains a pointer to a data array for the buffer, whose size must be at least as 
big as the size of a disk block, and a status field that summarizes the current status 
of the buffer. The status of a buffer is a combination of the following conditions: 

• The buffer is currently locked (the terms “locked” and “busy” will be used 
interchangeably, as will “free” and “unlocked”), 

• The buffer contains valid data, 

• The kernel must write the buffer contents to disk before reassigning the buffer; 
this condition is known as “delayed-write,” 

• The kernel is currently reading or writing the contents of the buffer to disk, 

• A process is currently waiting for the buffer to become free. 

The buffer header also contains two sets of pointers, used by the buffer allocation 
algorithms to maintain the overall structure of the buffer pool, as explained in the 
next section. 



3.2 STRUCTURE OF THE BUFFER POOL 

The kernel caches data in the buffer pool according to a least recently used 
algorithm: after it allocates a buffer to a disk block, it cannot use the buffer for 
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before 

after 




Figure 3.2. Free List of Buffers 



another block until all other buffers have been used more recently. The kernel 
maintains a free list of buffers that preserves the least recently used order. The 
free list is a doubly linked circular list of buffers with a dummy buffer header that 
marks its beginning and end (Figure 3.2). Every buffer is put on the free list when 
the system is booted. The kernel takes a buffer from the head of the free list when 
it wants any free buffer, but it can take a buffer from the middle of the free list if 
it identifies a particular block in the buffer pool. In both cases, it removes the 
buffer from the free list. When the kernel returns a buffer to the buffer pool, it 
usually attaches the buffer to the tail of the free list, occasionally to the head of the 
free list (for error cases), but never to the middle. As the kernel removes buffers 
from the free list, a buffer with valid data moves closer and closer to head of the 
free list (Figure 3.2). Hence, the buffers that are closer to the head of the free list 
have not been used as recently as those that are further from the head of the free 
list. 

When the kernel accesses a disk block, it searches for a buffer with the 
appropriate device-block number combination. Rather than search the entire buffer 
pool, it organizes the buffers into separate queues, hashed as a function of the 
device and block number. The kernel links the buffers on a hash queue into a 
circular, doubly linked list, similar to the structure of the free list. The number of 
buffers on a hash queue varies during the lifetime of the system, as will be seen. 
The kernel must use a hashing function that distributes the buffers uniformly across 
the set of hash queues, yet the hash function must be simple so that performance 
does not suffer. System administrators configure the number of hash queues when 
generating the operating system. 
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hash queue headers 




Figure 3.3. Buffers on the Hash Queues 



Figure 3.3 shows buffers on their hash queues: the headers of the hash queues 
are on the left side of the figure, and the squares on each row are buffers on a hash 
queue. Thus, squares marked 28, 4, and 64 represent buffers on the hash queue for 
“blkno 0 mod 4” (block number 0 modulo 4). The dotted lines between the buffers 
represent the forward and back pointers for the hash queue; for simplicity, later 
figures in this chapter will not show these pointers, but their existence is implicit. 
Similarly, the figure identifies blocks only by their block number, and it uses a hash 
function dependent only on a block number; however, implementations use the 
device number, too. 

Each buffer always exists on a hash queue, but there is no significance to its 
position on the queue. As stated above, no two buffers may simultaneously contain 
the contents of the same disk block; therefore, every disk block in the buffer pool 
exists on one and only one hash queue and only once on that queue. However, a 
buffer may be on the free list as well if its status is free. Because a buffer may be 
simultaneously on a hash queue and on the free list, the kernel has two ways to find 
it: It searches the hash queue if it is looking for a particular buffer, and it removes 
a buffer from the free list if it is looking for any free buffer. The next section will 
show how the kernel finds particular disk blocks in the buffer cache, and how it 
manipulates buffers on the hash queues and on the free list. To summarize, a 
buffer is always on a hash queue, but it may or may not be on the free list. 



3.3 SCENARIOS FOR RETRIEVAL OF A BUFFER 

As seen in Figure 2.1, high-level kernel algorithms in the file subsystem invoke the 
algorithms for managing the buffer cache. The high-level algorithms determine the 
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logical device number and block number that they wish to access when they 
attempt to retrieve a block. For example, if a process wants to read data from a 
file, the kernel determines which file system contains the file and which block in the 
file system contains the data, as will be seen in Chapter 4. When about to read 
data from a particular disk block, the kernel checks whether the block is in the 
buffer pool and, if it is not there, assigns it a free buffer. When about to write data 
to a particular disk block, the kernel checks whether the block is in the buffer pool, 
and if not, assigns a free buffer for that block. The algorithms for reading and 
writing disk blocks use the algorithm getblk (Figure 3.4) to allocate buffers from 
the pool. 

This section describes five typical scenarios the kernel may follow in getblk to 
allocate a buffer for a disk block. 

1. The kernel finds the block on its hash queue, and its buffer is free. 

2. The kernel cannot find the block on the hash queue, so it allocates a buffer 
from the free list. 

3. The kernel cannot find the block on the hash queue and, in attempting to 
allocate a buffer from the free list (as in scenario 2), finds a buffer on the 
free list that has been marked “delayed write.” The kernel must write the 
“delayed write” buffer to disk and allocate another buffer. 

4. The kernel cannot find the block on the hash queue, and the free list of 
buffers is empty. 

5. The kernel finds the block on the hash queue, but its buffer is currently busy. 

Let us now discuss each scenario in greater detail. 

When searching for a block in the buffer pool by its device-block number 
combination, the kernel finds the hash queue that should contain the block. It 
searches the hash queue, following the linked list of buffers until (in the first 
scenario) it finds the buffer whose device and block number match those for which 
it is searching. The kernel checks that the buffer is free and, if so, marks the 
buffer “busy” so that other processes 2 cannot access it. The kernel then removes 
the buffer from the free list, because a buffer cannot be both busy and on the free 
list. If other processes attempt to access the block while the buffer is busy, they 
sleep until the buffer is released, as will be seen. Figure 3.5 depicts the first 
scenario, where the kernel searches for block 4 on the hash queue marked “blkno 0 
mod 4.” Finding the buffer, the kernel removes it from the free list, leaving blocks 
5 and 28 adjacent on the free list. 



2. Recall from the last chapter that all kernel operations are done in the context of a process that is 
executing in kernel mode. Thus, the term “other processes” means that they are also executing in 
kernel mode. This term will be used when describing the interaction of several processes executing in 
kernel mode; if there is no interprocess interaction, the term “kernel” will be used 
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algorithm gctblk 
input: file system number 

block number 

output: locked buffer that can now be used for block 

( 

while (buffer not found) 

( 

if (block in hash queue) 

( 

if (buffer busy) /* scenario 5 */ 

( 

sleep (event buffer becomes free); 
continue; /* back to while loop */ 

) 

mark buffer busy; /* scenario 1 •/ 
remove buffer from free list; 
return buffer; 

} 

else /• block not on hash queue */ 

I 

if (there are no buffers on free list) /• scenario 4 •/ 

( 

sleep (event any buffer becomes free); 
continue; /* back to while loop *1 

) 

remove buffer from free list; 

if (buffer marked for delayed write) ( /* scenario 3 */ 

asynchronous write buffer to disk; 
continue; /• back to while loop •/ 

» 

/• scenario 2 — found a free buffer •/ 
remove buffer from old hash queue; 
put buffer onto new hash queue; 
return buffer; 

) 

) 

) 



Figure 3.4. Algorithm for Buffer Allocation 
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(a) Search for Block 4 on First Hash Queue 



hash queue headers 




(b) Remove Block 4 from Free List 



Figure 3.5. Scenario 1 in Finding a Buffer: Buffer on Hash Queue 
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algorithm brelse 
input: locked buffer 

output: none 

( 

wakeup all procs: event, waiting for any buffer to become free; 
wakeup all procs: event, waiting for this buffer to become free; 
raise processor execution level to block interrupts; 
if (buffer contents valid and buffer not old) 
enqueue buffer at end of free list 

else 

enqueue buffer at beginning of free list 
lower processor execution level to allow interrupts; 
unlock (buffer); 

) 



Figure 3.6. Algorithm for Releasing a Buffer 



Before continuing to the other scenarios, let us consider what happens to a 
buffer after it is allocated. The kernel may read data from the disk to the buffer 
and manipulate it or write data to the buffer and possibly to the disk. The kernel 
leaves the buffer marked busy; no other process can access it and change its 
contents while it is busy, thus preserving the integrity of the data in the buffer. 
When the kernel finishes using the buffer, it releases the buffer according to 
algorithm brelse (Figure 3.6). It wakes up processes that had fallen asleep because 
the buffer was busy and processes that had fallen asleep because no buffers 
remained on the free list. In both cases, release of a buffer means that the buffer is 
available for use by the sleeping processes, although the first process that gets the 
buffer locks it and prevents the other processes from getting it (recall Section 
2.2.2 A). The kernel places the buffer at the end of the free list, unless an I/O 
error occurred or unless it specifically marked the buffer “old,” as will be seen later 
in this chapter; in the latter cases, it places the buffer at the beginning of the free 
list. The buffer is now free for another process to claim it. 

Just as the kernel invokes algorithm brelse when a process has no more need for 
a buffer, it also invokes the algorithm when handling a disk interrupt to release 
buffers used for asynchronous I/O to and from the disk, as will be seen in Section 
3.4. The kernel raises the processor execution level to prevent disk interrupts while 
manipulating the free list, thereby preventing corruption of the buffer pointers that 
could result from a nested call to brelse. Similar bad effects could happen if an 
interrupt handler invoked brelse while a process was executing getblk , so the kernel 
raises the processor execution level at strategic places in getblk , too. The exercises 
explore these cases in greater detail. 

In the second scenario in algorithm getblk , the kernel searches the hash queue 
that should contain the block but fails to find it there. Since the block cannot be 
on another hash queue because it cannot “hash” elsewhere, it is not in the buffer 
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cache. So the kernel removes the first buffer from the free list instead; that buffer 
had been allocated to another disk block and is also on a hash queue. If the buffer 
has not been marked for a delayed write (as will be described later), the kernel 
marks the buffer busy, removes it from the hash queue where it currently resides, 
reassigns the buffer header’s device and block number to that of the disk block for 
which the process is searching, and places the buffer on the correct hash queue. 
The kernel uses the buffer but has no record that the buffer formerly contained 
data for another disk block. A process searching for the old disk block will not find 
it in the pool and will have to allocate a new buffer for it from the free list, exactly 
as outlined here. When the kernel finishes with the buffer, it releases it as 
described above. In Figure 3.7, for example, the kernel searches for block 18 but 
does not find it on the hash queue marked “blkno 2 mod 4.” It therefore removes 
the first buffer from the free list (block 3), assigns it to block 18, and places it on 
the appropriate hash queue. 

In the third scenario in algorithm getblk , the kernel also has to allocate a buffer 
from the free list. However, it discovers that the buffer it removes from the free 
list has been marked for “delayed write,” so it must write the contents of the buffer 
to disk before using the buffer. The kernel starts an asynchronous write to disk and 
tries to allocate another buffer from the free list. When the asynchronous write 
completes, the kernel releases the buffer and places it at the head of the free list. 
The buffer had started at the end of the free list and had traveled to the head of 
the free list. If, after the asynchronous write, the kernel were to place the buffer at 
the end of the free list, the buffer would get a free trip through the free list, 
working against the least recently used algorithm. For example, in Figure 3.8, the 
kernel cannot find block 18, but when it attempts to allocate the first two buffers 
(one at a time) on the free list, it finds them marked for delayed write. The kernel 
removes them from the free list, starts write operations to disk for the blocks, and 
allocates the third buffer on the free list, block 4. It reassigns the buffer’s device 
and block number fields appropriately and places the buffer, now marked block 18, 
on its new hash queue. 

In the fourth scenario (Figure 3.9), the kernel, acting for process A, cannot find 
the disk block on its hash queue, so it attempts to allocate a new buffer from the 
free list, as in the second scenario. However, no buffers are available on the free 
list, so process A goes to sleep until another process executes algorithm brelse , 
freeing a buffer. When the kernel schedules process A, it must search the hash 
queue again for the block. It cannot allocate a buffer immediately from the free 
list, because it is possible that several processes were waiting for a free buffer and 
that one of them allocated a newly freed buffer for the target block sought by 
process A. Thus, searching for the block again insures that only one buffer 
contains the disk block. Figure 3.10 depicts the contention between two processes 
for a free buffer. 

The final scenario (Figure 3.11) is complicated, because it involves complex 
relationships between several processes. Suppose the kernel, acting for process A, 
searches for a disk block and allocates a buffer but goes to sleep before freeing the 
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(a) Search for Block 18, Delayed Write Blocks on Free List 



hash queue headers 




(b) Writing Blocks 3, 5, Reassign 4 to 18 
Figure 3.8. Third Scenario for Buffer Allocation 
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hash queue headers 
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Search for Block 18, Empty Free List 
Figure 3.9. Fourth Scenario for Allocating Buffer 



buffer. For example, if process A attempts to read a disk block and allocates a 
buffer as in scenario 2, then it will sleep while it waits for the I/O transmission 
from disk to complete. While process A sleeps, suppose the kernel schedules a 
second process, B, which tries to access the disk block whose buffer was just locked 
by process A. Process B (going through scenario 5) will find the locked block on 
the hash queue. Since it is illegal to use a locked buffer and it is illegal to allocate 
a second buffer for a disk block, process B marks the buffer “in demand” and then 
sleeps and waits for process A to release the buffer. 

Process A will eventually release the buffer and notice that the buffer is in 
demand. It awakens all processes sleeping on the event “the buffer becomes free,” 
including process B. When the kernel again schedules process B, process B must 
verify that the buffer is free. Another process, C, may have been waiting for the 
same buffer, and the kernel may have scheduled C to run before process B; process 
C may have gone to sleep leaving the buffer locked. Hence, process B must check 
that the block is indeed free. 

Process B must also verify that the buffer contains the disk block that it 
originally requested, because process C may have allocated the buffer to another 
block, as in scenario 2. When process B executes, it may find that it had been 
waiting for the wrong buffer, so it must search for the block again: If it were to 
allocate a buffer automatically from the free list, it would miss the possibility that 
another process just allocated a buffer for the block. 
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Process A 



Process B 



Cannot find block b : 

on hash queue : 

No buffers on free list • 

Sleep : 

: Cannot find block b 

: on hash queue 

: No buffers on free list 

• Sleep 

Somebody frees a buffer: brelsc 



V 

Time 



Takes buffer from free list 
Assign to block b 



Figure 3.10. Race for Free Buffer 



In the end, process B will find its block, possibly allocating a new buffer from 
the free list as in the second scenario. In Figure 3.11, for example, a process 
searching for block 99 finds it on its hash queue, but the block is marked busy. 
The process sleeps until the block becomes free and then restarts the algorithm 
from the beginning. Figure 3.12 depicts the contention for a locked buffer. 

The algorithm for buffer allocation must be safe; processes must not sleep 
forever, and they must eventually get a buffer. The kernel guarantees that all 
processes waiting for buffers will wake up, because it allocates buffers during the 
execution of system calls and frees them before returning. 3 Processes in user mode 
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Figure 3.11. Fifth Scenario for Buffer Allocation 



do not control the allocation of kernel buffers directly, so they cannot purposely 
“hog” buffers. The kernel loses control over a buffer only when it waits for the 
completion of I/O between the buffer and the disk. It is conceivable that a disk 
drive is corrupt so that it cannot interrupt the CPU, preventing the kernel from 
ever releasing the buffer. The disk driver must monitor the hardware for such 
cases and return an error to the kernel for a bad disk job. In short, the kernel can 
guarantee that processes sleeping for a buffer will wake up eventually. 

It is also possible to imagine cases where a process is starved out of accessing a 
buffer. In the fourth scenario, for example, if several processes sleep while waiting 
for a buffer to become free, the kernel does not guarantee that they get a buffer in 
the order that they requested one. A process could sleep and wake up when a 
buffer becomes free, only to go to sleep again because another process got control of 
the buffer first. Theoretically, this could go on forever, but practically, it is not a 
problem because of the many buffers that are typically configured in the system. 



3. The mount system call is an exception, because it allocates a buffer until a later umount call. This 
exception is not critical, because the total number of buffers far exceeds the number of active 
mounted file systems. 
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Process A 



Process B 
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Allocate buffer 
to block b 

Lock buffer 

Initiate I/O 



Sleep until I/O done : 

: Find block b 
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buffer docs not contain 
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Figure 3.12. Race for a Locked Buffer 



3.4 READING AND WRITING DISK BLOCKS 

Now that the buffer allocation algorithm has been covered, the procedures for 
reading and writing disk blocks should be easy to understand. To read a disk block 
(Figure 3.13), a process uses algorithm getblk to search for it in the buffer cache. 
If it is in the cache, the kernel can return it immediately without physically reading 
the block from the disk. If it is not in the cache, the kernel calls the disk driver to 
“schedule” a read request and goes to sleep awaiting the event that the I/O 
completes. The disk driver notifies the disk controller hardware that it wants to 
read data, and the disk controller later transmits the data to the buffer. Finally, 
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algorithm bread /• block read •/ 
input: file system block number 
output: buffer containing data 

l 

get buffer for block (algorithm getblk): 
if (buffer data valid) 
return buffer; 
initiate disk read; 
sleep (event disk read complete); 
return (buffer); 

} 



Figure 3.13. Algorithm for Reading a Disk Block 



the disk controller interrupts the processor when the I/O is complete, and the disk 
interrupt handler awakens the sleeping process; the contents of the disk block are 
now in the buffer. The modules that requested the particular block now have the 
data; when they no longer need the buffer they release it so that other processes can 
access it. 

Chapter 5 shows how higher-level kernel modules (such as the file subsystem) 
may anticipate the need for a second disk block when a process reads a file 
sequentially. The modules request the second I/O asynchronously in the hope that 
the data will be in memory when needed, improving performance. To do this, the 
kernel executes the block read-ahead algorithm breada (Figure 3.14): The kernel 
checks if the first block is in the cache and, if it is not there, invokes the disk driver 
to read that block. If the second block is not in the buffer cache, the kernel 
instructs the disk driver to read it asynchronously. Then the process goes to sleep 
awaiting the event that the I/O is complete on the first block. When it awakens, it 
returns the buffer for the first block, and does not care when the I/O for the second 
block completes. When the I/O for the second block does complete, the disk 
controller interrupts the system; the interrupt handler recognizes that the I/O was 
asynchronous and releases the buffer (algorithm brelse). If it would not release the 
buffer, the buffer would remain locked and, therefore, inaccessible to all processes. 
It is impossible to unlock the buffer beforehand, because I/O to the buffer was 
active, and hence the buffer contents were not valid. Later, if the process wants to 
read the second block, it should find it in the buffer cache, the I/O having 
completed in the meantime. If, at the beginning of breada , the first block was in 
the buffer cache, the kernel immediately checks if the second block is in the cache 
and proceeds as just described. 

The algorithm for writing the contents of a buffer to a disk block is similar 
(Figure 3.15). The kernel informs the disk driver that it has a buffer whose 
contents should be output, and the disk driver schedules the block for I/O. If the 
write is synchronous, the calling process goes to sleep awaiting I/O completion and 
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algorithm breada /* block read and read ahead */ 
input: (1) file system block number for immediate read 

(2) file system block number for asynchronous read 
output: buffer containing data for immediate read 
{ 

if (first block not in cache) 

( 

get buffer for first block (algorithm getblk); 
if (buffer data not valid) 
initiate disk read; 

} 

if (second block not in cache) 

( 

get buffer for second block (algorithm getblk); 
if (buffer data valid) 

release buffer (algorithm brelse); 
else 

initiate disk read; 

} 

if (first block was originally in cache) 

{ 

read first block (algorithm bread); 
return buffer; 

) 

sleep (event first buffer contains valid data); 
return buffer; 

} 



Figure 3.14. Algorithm for Block Read Ahead 



releases the buffer when it awakens. If the write is asynchronous, the kernel starts 
the disk write but does not wait for the write to complete. The kernel will release 
the buffer when the I/O completes. 

There are occasions, described in the next two chapters, when the kernel does 
not write data immediately to disk. If it docs a “delayed write,” it marks the 
buffer accordingly, releases the buffer using algorithm brelse , and continues without 
scheduling I/O. The kernel writes the block to disk before another process can 
reallocate the buffer to another block, as described in scenario 3 of getblk. In the 
meantime, the kernel hopes that a process accesses the block before the buffer must 
be written to disk; if that process subsequently changes the contents of the buffer, 
the kernel saves an extra disk operation. 

A delayed write is different from an asynchronous write. When doing an 
asynchronous write, the kernel starts the disk operation immediately but does not 
wait for its completion. For a “delayed write,” the kernel puts off the physical 
write to disk as long as possible; then, recalling the third scenario in algorithm 
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algorithm bwrite /• block write •/ 
input: buffer 

output: none 

{ 

initiate disk write; 
if (I/O synchronous) 

( 

sleep(event I/O complete); 
release buffer (algorithm brelse); 

} 

else if (buffer marked for delayed write) 

mark buffer to put at head of free list; 

} 



Figure 3.15. Algorithm for Writing a Disk Block 



getblk , it marks the buffer “old” and writes the block to disk asynchronously. The 
disk controller later interrupts the system and releases the buffer, using algorithm 
brelse\ the buffer ends up on the head of the free list, because it was “old.” 
Because of the two asynchronous I/O operations — block read ahead and delayed 
write — the kernel can invoke brelse from an interrupt handler. Hence, it must 
prevent interrupts in any procedure that manipulates the buffer free list, because 
brelse places buffers on the free list. 



3.5 ADVANTAGES AND DISADVANTAGES OF THE BUFFER CACHE 

Use of the buffer cache has several advantages and, unfortunately, some 
disadvantages. 

• The use of buffers allows uniform disk access, because the kernel does not need 
to know the reason for the I/O. Instead, it copies data to and from buffers, 
regardless of whether the data is part of a file, an inode, or a super block. The 
buffering of disk I/O makes the code more modular, since the parts of the 
kernel that do the I/O with the disk have one interface for all purposes. In 
short, system design is simpler. 

• The system places no data alignment restrictions on user processes doing I/O, 
because the kernel aligns data internally. Hardware implementations frequently 
require a particular alignment of data for disk I/O, such as aligning the data on 
a two-byte boundary or on a four-byte boundary in memory. Without a buffer 
mechanism, programmers would have to make sure that their data buffers were 
correctly aligned. Many programmer errors would result, and programs would 
not be portable to UNIX systems running on machines with stricter address 
alignment properties. By copying data from user buffers to system buffers (and 
vice versa), the kernel eliminates the need for special alignment of user buffers, 
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making user programs simpler and more portable. 

• Use of the buffer cache can reduce the amount of disk traffic, thereby increasing 
overall system throughput and decreasing response time. Processes reading 
from the file system may find data blocks in the cache and avoid the need for 
disk I/O. The kernel frequently uses “delayed write” to avoid unnecessary disk 
writes, leaving the block in the buffer cache and hoping for a cache hit on the 
block. Obviously, the chances of a cache hit are greater for systems with many 
buffers. However, the number of buffers a system can profitably configure is 
constrained by the amount of memory that should be kept available for 
executing processes: if too much memory is used for buffers, the system may 
slow down because of excessive process swapping or paging. 

• The buffer algorithms help insure file system integrity, because they maintain a 
common, single image of disk blocks contained in the cache. If two processes 
simultaneously attempt to manipulate one disk block, the buffer algorithms 
( getblk for example) serialize their access, preventing data corruption. 

• Reduction of disk traffic is important for good throughput and response time, 
but the cache strategy also introduces several disadvantages. Since the kernel 
does not immediately write data to the disk for a delayed write, the system is 
vulnerable to crashes that leave disk data in an incorrect state. Although recent 
system implementations have reduced the damage caused by catastrophic 
events, the basic problem remains: A user issuing a write system call is never 
sure when the data finally makes its way to disk. 4 

• Use of the buffer cache requires an extra data copy when reading and writing to 
and from user processes. A process writing data copies the data into the kernel, 
and the kernel copies the data to disk; a process reading data has the data read 
from disk into the kernel and from the kernel to the user process. When 
transmitting large amounts of data, the extra copy slows down performance, but 
when transmitting small amounts of data, it improves performance because the 
kernel buffers the data (using algorithms getblk and delayed write) until it is 
economical to transmit to or from the disk. 



3.6 SUMMARY 

This chapter has presented the structure of the buffer cache and the various 
methods by which the kernel locates blocks in the cache. The buffer algorithms 
combine several simple ideas to provide a sophisticated caching mechanism. The 
kernel uses the least-recently-used replacement algorithm to keep blocks in the 



4. The standard I/O package available to C language programs includes an fflush call. This function 
call flushes data from buffers in the user address space (part of the package) into the kernel. 
However, the user still does not know when the kernel writes the data to the disk. 
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buffer cache, assuming that blocks that were recently accessed are likely to be 
accessed again soon. The order that the buffers appear on the free list specifies the 
order in which they were last used. Other buffer replacement algorithms, such as 
first-in-first-out or least-frequently-used, are either more complicated to implement 
or result in lower cache hit ratios. The hash function and hash queues enable the 
kernel to find particular blocks quickly, and use of doubly linked lists makes it easy 
to remove buffers from the lists. 

The kernel identifies the block it needs by supplying a logical device number 
and block number. The algorithm getblk searches the buffer cache for a block and, 
if the buffer is present and free, locks the buffer and returns it. If the buffer is 
locked, the requesting process sleeps until it becomes free. The locking mechanism 
ensures that only one process at a time manipulates a buffer. If the block is not in 
the cache, the kernel reassigns a free buffer to the block, locks it and returns it 
The algorithm bread allocates a buffer for a block and reads the data into the 
buffer, if necessary. The algorithm bwrite copies data into a previously allocated 
buffer. If, in execution of certain higher-level algorithms, the kernel determines 
that it is not necessary to copy the data immediately to disk, it marks the buffer 
“delayed write” to avoid unnecessary I/O. Unfortunately, the “delayed write” 
scheme means that a process is never sure when the data is physically on disk. If 
the kernel writes data synchronously to disk, it invokes the disk driver to write the 
block to the file system and waits for an I/O completion interrupt. 

The kernel uses the buffer cache in many ways. It transmits data between 
application programs and the file system via the buffer cache, and it transmits 
auxiliary system data such as inodes between higher-level kernel algorithms and the 
file system. It also uses the buffer cache when reading programs into memory for 
execution. The following chapters will describe many algorithms that use the 
procedures described in this chapter. Other algorithms that cache inodes and pages 
of memory also use techniques similar to those described for the buffer cache. 



3.7 EXERCISES 

1. Consider the hash function in Figure 3.3. The best hash function is one that 
distributes the blocks uniformly over the set of hash queues. What would be an 
optimal hashing function? Should a hash function use the logical device number in its 
calculations? 

2. In the algorithm getblk , if the kernel removes a buffer from the free list, it must raise 
the processor priority level to block out interrupts before checking the free list. Why? 

* 3. In algorithm getblk , the kernel must raise the processor priority level to block out 
interrupts before checking if a block is busy. (This is not shown in the text.) Why? 

4. In algorithm bruise* the kernel enqueues the buffer at the head of the free list if the 
buffer contents are invalid. If the contents are invalid, should the buffer appear on a 
hash queue? 

5. Suppose the kernel does a delayed write of a block. What happens when another 
process takes that block from its hash queue? From the free list? 
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* 6. If several processes contend for a buffer, the kernel guarantees that none of them sleep 

forever, but it does not guarantee that a process will not be starved out from use of a 
buffer. Redesign getblk so that a process is guaranteed eventual use of a buffer. 

7. Redesign the algorithms for getblk and brelse such that the kernel does not follow a 
least-recently-used scheme but a first-in-first-out scheme. Repeat this problem using a 
least-frequently-used scheme. 

8. Describe a scenario where the buffer data is already valid in algorithm bread. 

* 9. Describe the various scenarios that can happen in algorithm breada. What happens 

on the next invocation of bread or breada when the current read-ahead block will be 
read? In algorithm breada , if the first or second block are not in the cache, the later 
test to see if the buffer data is valid implies that the block could be in the buffer pool. 
How is this possible? 

10. Describe an algorithm that asks for and receives any free buffer from the buffer pool. 
Compare this algorithm to getblk. 

11. Various system calls such as umount and sync (Chapter 5) require the kernel to flush 
to disk all buffers that are “delayed write” for a particular file system. Describe an 
algorithm that implements a buffer flush. What happens to the order of buffers on the 
free list as a result of the flush operation? How can the kernel be sure that no other 
process sneaks in and writes a buffer with delayed write to the file system while the 
flushing process sleeps waiting for an I/O completion? 

12. Define system response time as the average time it takes to complete a system call. 
Define system throughput as the number of processes the system can execute in a 
given time period. Describe how the buffer cache can help response time. Does it 
necessarily help system throughput? 
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As observed in Chapter 2, every file on a UNIX system has a unique inode. The 
inode contains the information necessary for a process to access a file, such as file 
ownership, access rights, file size, and location of the file’s data in the file system. 
Processes access files by a well defined set of system calls and specify a file by a 
character string that is the path name. Each path name uniquely specifies a file, 
and the kernel converts the path name to the file’s inode. 

This chapter describes the internal structure of files in the UNIX system, and 
the next chapter describes the system call interface to files. Section 4.1 examines 
the inode and how the kernel manipulates it, and Section 4.2 examines the internal 
structure of regular files and how the kernel reads and writes their data. Section 
4.3 investigates the structure of directories, the files that allow the kernel to 
organize the file system as a hierarchy of files, and Section 4.4 presents the 
algorithm for converting user file names to inodes. Section 4.5 gives the structure 
of the super block, and Sections 4.6 and 4.7 present the algorithms for assignment 
of disk inodes and disk blocks to files. Finally, Section 4.8 talks about other file 
types in the system, namely, pipes and device files. 

The algorithms described in this chapter occupy the layer above the buffer 
cache algorithms explained in the last chapter (Figure 4.1). The algorithm iget 
returas a previously identified inode, possibly reading it from disk via the buffer 
cache, and the algorithm iput releases the inode. The algorithm bntap sets kernel 
parameters for accessing a file. The algorithm namei converts a user-level path 
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Lower Level File System Algorithms 



namei 


alloc free 


ialloc ifree 


iget iput bmap 


buffer allocation algorithms 


getblk brelse bread breada bwrite 



Figure 4.1. File System Algorithms 



name to an inode, using the algorithms feet, iput> and bmap. Algorithms alloc and 
free allocate and free disk blocks for files, and algorithms ialloc and ifree assign 
and free inodes for files. 



4.1 INODES 



4.1.1 Definition 

Inodes exist in a static form on disk, and the kernel reads them into an in-core 
inode to manipulate them. Disk inodes consist of the following fields: 

• File owner identifier. Ownership is divided between an individual owner and a 
“group” owner and defines the set of users who have access rights to a file. The 
superuser has access rights to all files in the system. 

• File type. Files may be of type regular, directory, character or block special, or 
FIFO (pipes). 

• File access permissions. The system protects files according to three classes: 
the owner and the group owner of the file, and other users; each class has access 
rights to read, write and execute the file, which can be set individually. Because 
directories cannot be executed, execution permission for a directory gives the 
right to search the directory for a file name. 

• File access times, giving the time the file was last modified, when it was last 
accessed, and when the inode was last modified. 





62 



INTERNAL REPRESENTATION OF FILES 



• Number of links to the file, representing the number of names the file has in the 
directory hierarchy. Chapter 5 explains file links in detail. 

• Table of contents for the disk addresses of data in a file. Although users treat 
the data in a file as a logical stream of bytes, the kernel saves the data in 
discontiguous disk blocks. The inode identifies the disk blocks that contain the 
file’s data. 

• File size. Data in a file is addressable by the number of bytes from the 
beginning of the file, starting from byte offset 0, and the file size is 1 greater 
than the highest byte offset of data in the file. For example, if a user creates a 
file and writes only 1 byte of data at byte offset 1000 in the file, the size of the 
file is 1001 bytes. 

The inode does not specify the path name(s) that access the file. 



owner mjb 
group os 
type regular file 
perms rwxr-xr-x 
accessed Oct 23 1984 1:45 P.M. 
modified Oct 22 1984 10:30 A.M. 
inode Oct 23 1984 1:30 P.M. 
size 6030 bytes 
disk addresses 



Figure 4.2. Sample Disk Inode 



Figure 4.2 shows the disk inode of a sample file. This inode is that of a 
regular file owned by “mjb,” which contains 6030 bytes. The system permits 
“mjb” to read, write, or execute the file; members of the group “os” and all other 
users can only read or execute the file, not write it. The last time anyone read the 
file was on October 23, 1984, at 1:45 in the afternoon, and the last time anyone 
wrote the file was on October 22, 1984, at 10:30 in the morning. The inode was 
last changed on October 23, 1984, at 1:30 in the afternoon, although the data in 
the file was not written at that time. The kernel encodes the above information in 
the inode. Note the distinction between writing the contents of an inode to disk 
and writing the contents of a file to disk. The contents of a file change only when 
writing it. The contents of an inode change when changing the contents of a file or 
when changing its owner, permission, or link settings. Changing the contents of a 
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file automatically implies a change to the inode, but changing the inode docs not 
imply that the contents of the file change. 

The in-core copy of the inode contains the following fields in addition to the 
fields of the disk inode: 

• The status of the in-core inode, indicating whether 

— the inode is locked, 

— a process is waiting for the inode to become unlocked, 

— the in-core representation of the inode differs from the disk copy as a result 
of a change to the data in the inode, 

— the in-core representation of the file differs from the disk copy as a result of 
a change to the file data, 

— the file is a mount point (Section 5.15). 

• The logical device number of the file system that contains the file. 

• The inode number. Since inodes are stored in a linear array on disk (recall 
Section 2.2.1), the kernel identifies the number of a disk inode by its position in 
the array. The disk inode docs not need this field. 

• Pointers to other in-core inodes. The kernel links inodes on hash queues and on 
a free list in the same way that it links buffers on buffer hash queues and on the 
buffer free list. A hash queue is identified according to the inode’s logical 
device number and inode number. The kernel can contain at most one in-core 
copy of a disk inode, but inodes can be simultaneously on a hash queue and on 
the free list. 

• A reference count, indicating the number of instances of the file that are active 
(such as when opened). 

Many fields in the in-core inode arc analogous to fields in the buffer header, and 
the management of inodes is similar to the management of buffers. The inode lock, 
when set, prevents other processes from accessing the inode; other processes set a 
flag in the inode when attempting to access it to indicate that they should be 
awakened when the lock is released. The kernel sets other flags to indicate 
discrepancies between the disk inode and the in-core copy. When the kernel needs 
to record changes to the file or to the inode, it writes the in-core copy of the inode 
to disk after examining these flags. 

The most striking difference between an in-core inode and a buffer header is the 
in-core reference count, which counts the number of active instances of the file. An 
inode is active when a process allocates it, such as when opening a file. An inode is 
on the free list only if its reference count is 0, meaning that the kernel can 
reallocate the in-core inode to another disk inode. The free list of inodes thus 
serves as a cache of inactive inodes: If a process attempts to access a file whose 
inode is not currently in the in-core inode pool, the kernel reallocates an in-core 
inode from the free list for its use. On the other hand, a buffer has no reference 
count; it is on the free list if and only if it is unlocked. 
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algorithm iget 

input: file system inode number 
output: locked inode 

i 

while (not done) 

( 

if (inode in inode cache) 

{ 

if (inode locked) 

( 

sleep (event inode becomes unlocked); 
continue; /* loop back to while •/ 

) 

/* special processing for mount points (Chapter 5) */ 
if (inode on inode free list) 
remove from free list; 
increment inode reference count; 
return (inode); 

} 

/* inode not in inode cache */ 
if (no inodes on free list) 
return (error); 

remove new inode from free list; 

reset inode number and file system; 

remove inode from old hash queue, place on new one; 

read inode from disk (algorithm bread); 

initialize inode (e.g. reference count to 1); 

return (inode); 

) 

1 



Figure 4.3. Algorithm for Allocation of In-Core Inodes 



4.1.2 Accessing Inodes 

The kernel identifies particular inodes by their file system and inode number and 
allocates in-core inodes at the request of higher-level algorithms. The algorithm 
iget allocates an in-core copy of an inode (Figure 4.3); it is almost identical to the 
algorithm getblk for finding a disk block in the buffer cache. The kernel maps the 
device number and inode number into a hash queue and searches the queue for the 
inode. If it cannot find the inode, it allocates one from the free list and locks it. 
The kernel then prepares to read the disk copy of the newly accessed inode into the 
in-core copy. It already knows the inode number and logical device and computes 
the logical disk block that contains the inode according to how many disk inodes fit 
into a disk block. The computation follows the formula 
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block num — ((inode number — 1) / number of inodes per block) + 

start block of inode list 

where the division operation returns the integer part of the quotient. For example, 
assuming that block 2 is the beginning of the inode list and that there are 8 inodes 
per block, then inode number 8 is in disk block 2, and inode number 9 is in disk 
block 3. If there are 16 inodes in a disk block, then inode numbers 8 and 9 are in 
disk block 2, and inode number 17 is the first inode in disk block 3. 

When the kernel knows the device and disk block number, it reads the block 
using the algorithm bread (Chapter 2), then uses the following formula to compute 
the byte offset of the inode in the block: 

((inode number — 1) modulo (number of inodes per block)) * size of disk inode 

For example, if each disk inode occupies 64 bytes and there are 8 inodes per disk 
block, then inode number 8 starts at byte offset 448 in the disk block. The kernel 
removes the in-core inode from the free list, places it on the correct hash queue, 
and sets its in-core reference count to 1. It copies the file type, owner fields, 
permission settings, link count, file size, and the table of contents from the disk 
inode to the in-core inode, and returns a locked inode. 

The kernel manipulates the inode lock and reference count independently. The 
lock is set during execution of a system call to prevent other processes from 
accessing the inode while it is in use (and possibly inconsistent). The kernel 
releases the lock at the conclusion of the system call: an inode is never locked 
across system calls. The kernel increments the reference count for every active 
reference to a file. For example. Section 5.1 will show that it increments the inode 
reference count when a process opens a file. It decrements the reference count only 
when the reference becomes inactive, for example, when a process closes a file. 
The reference count thus remains set across multiple system calls. The lock is free 
between system calls to allow processes to share simultaneous access to a file; the 
reference count remains set between system calls to prevent the kernel from 
reallocating an active in-core inode. Thus, the kernel can lock and unlock an 
allocated inode independent of the value of the reference count. System calls other 
than open allocate and release inodes, as will be seen in Chapter 5. 

Returning to algorithm iget , if the kernel attempts to take an inode from the 
free list but finds the free list empty, it reports an error. This is different from the 
philosophy the kernel follows for disk buffers, where a process sleeps until a buffer 
becomes free: Processes have control over the allocation of inodes at user level via 
execution of open and close system calls, and consequently the kernel cannot 
guarantee when an inode will become available. Therefore, a process that goes to 
sleep waiting for a free inode to become available may never wake up. Rather than 
leave such a process “hanging,” the kernel fails the system call. However, 
processes do not have such control over buffers: Because a process cannot keep a 
buffer locked across system calls, the kernel can guarantee that a buffer will 
become free soon, and a process therefore sleeps until one is available. 
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The preceding paragraphs cover the case where the kernel allocated an inode 
that was not in the inode cache. If the inode is in the cache, the process (A) would 
find it on its hash queue and check if the inode was currently locked by another 
process (B). If the inode is locked, process A sleeps, setting a flag in the in-core 
inode to indicate that it is waiting for the inode to become free. When process B 
later unlocks the inode, it awakens all processes (including process A) waiting for 
the inode to become free. When process A is finally able to use the inode, it locks 
the inode so that other processes cannot allocate it. If the reference count was 
previously 0, the inode also appears on the free list, so the kernel removes it from 
there: the inode is no longer free. The kernel increments the inode reference count 
and returns a locked inode. 

To summarize, the iget algorithm is used toward the beginning of system calls 
when a process first accesses a file. The algorithm returns a locked inode structure 
with reference count 1 greater than it had previously been. The in-core inode 
contains up-to-date information on the state of the file. The kernel unlocks the 
inode before returning from the system call so that other system calls can access 
the inode if they wish. Chapter 5 treats these cases in greater detail. 



algorithm iput /• release (put) access to in— core inode •/ 

input: pointer to in— core inode 
output: none 
( 

lock inode if not already locked; 
decrement inode reference count; 
if (reference count 0) 

( 

if (inode link count “ 0) 

{ 

free disk blocks for file (algorithm free, section 4.7); 
set file type to 0; 

free inode (algorithm ifree, section 4.6); 

} 

if (file accessed or inode changed or file changed) 
update disk inode; 
put inode on free list; 

} 

release inode lock; 

) 



Figure 4.4. Releasing an Inode 
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4.1.3 Releasing Inodes 

When the kernel releases an inode (algorithm iput , Figure 4.4), it decrements its 
in-core reference count. If the count drops to 0, the kernel writes the inode to disk 
if the in-core copy differs from the disk copy. They differ if the file data has 
changed, if the file access time has changed, or if the file owner or access 
permissions have changed. The kernel places the inode on the free list of inodes, 
effectively caching the inode in case it is needed again soon. The kernel may also 
release all data blocks associated with the file and free the inode if the number of 
links to the file is 0. 



4.2 STRUCTURE OF A REGULAR FILE 

As mentioned above, the inode contains the table of contents to locate a file’s data 
on disk. Since each block on a disk is addressable by number, the table of contents 
consists of a set of disk block numbers. If the data in a file were stored in a 
contiguous section of the disk (that is, the file occupied a linear sequence of disk 
blocks), then storing the start block address and the file size in the inode would 
suffice to access all the data in the file. However, such an allocation strategy would 
not allow for simple expansion and contraction of files in the file system without 
running the risk of fragmenting free storage area on the disk. Furthermore, the 
kernel would have to allocate and reserve contiguous space in the file system before 
allowing operations that would increase the file size. 
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Figure 4.5. Allocation of Contiguous Files and Fragmentation of Free Space 



For example, suppose a user creates three files. A, B and C, each consisting of 
10 disk blocks of storage, and suppose the system allocated storage for the three 
files contiguously. If the user then wishes to add 5 blocks of data to the middle file, 
B, the kernel would have to copy file B to a place in the file system that had room 
for 15 blocks of storage. Aside from the expense of such an operation, the disk 
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blocks previously occupied by file B’s data would be unusable except for files 
smaller than 10 blocks (Figure 4.5). The kernel could minimize fragmentation of 
storage space by periodically running garbage collection procedures to compact 
available storage, but that would place an added drain on processing power. 

For greater flexibility, the kernel allocates file space one block at a time and 
allows the data in a file to be spread throughout the file system. But this allocation 
scheme complicates the task of locating the data. The table of contents could 
consist of a list of block numbers such that the blocks contain the data belonging to 
the file, but simple calculations show that a linear list of file blocks in the inode is 
difficult to manage. If a logical block contaias IK bytes, then a file consisting of 
10K bytes would require an index of 10 block numbers, but a file containing 100K 
bytes would require an index of 100 block numbers. Either the size of the inode 
would vary according to the size of the file, or a relatively low limit would have to 
be placed on the size of a file. 

To keep the inode structure small yet still allow large files, the table of contents 
of disk blocks conforms to that shown in Figure 4.6. The System V UNIX system 
runs with 13 entries in the inode table of contents, but the principles are 
independent of the number of entries. The blocks marked “direct” in the figure 
contain the numbers of disk blocks that contain real data. The block marked 
“single indirect” refers to a block that contains a list of direct block numbers. To 
access the data via the indirect block, the kernel must read the indirect block, find 
the appropriate direct block entry, and then read the direct block to find the data. 
The block marked “double indirect” contains a list of indirect block numbers, and 
the block marked “triple indirect” contains a list of double indirect block numbers. 

In principle, the method could be extended to support “quadruple indirect 
blocks,” “quintuple indirect blocks,” and so on, but the current structure has 
sufficed in practice. Assume that a logical block on the file system holds IK bytes 
and that a block number is addressable by a 32 bit (4 byte) integer. Then a block 
can hold up to 256 block numbers. The maximum number of bytes that could be 
held in a file is calculated (Figure 4.7) at well over 16 gigabytes, using 10 direct 
blocks and 1 indirect, 1 double indirect, and 1 triple indirect block in the inode. 
Given that the file size field in the inode is 32 bits, the size of a file is effectively 
limited to 4 gigabytes (2 32 ). 

Processes access data in a file by byte offset. They work in terms of byte counts 
and view a file as a stream of bytes starting at byte address 0 and going up to the 
size of the file. The kernel converts the user view of bytes into a view of blocks: 
The file starts at logical block 0 and continues to a logical block number 
corresponding to the file size. The kernel accesses the inode and converts the 
logical file block into the appropriate disk block. Figure 4.8 gives the algorithm 
bmap for converting a file byte offset into a physical disk block. 

Consider the block layout for the file in Figure 4.9 and assume that a disk block 
contains 1024 bytes. If a process wants to access byte offset 9000, the kernel 
calculates that the byte is in direct block 8 in the file (counting from 0). It then 
accesses block number 367; the 808th byte in that block (starting from 0) is byte 
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Figure 4.6. Direct and Indirect Blocks in Inode 
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10 direct blocks with IK bytes each — 10K bytes 

1 indirect block with 256 direct blocks - 256K bytes 

1 double indirect block with 256 indirect blocks - 64M bytes 

1 triple indirect block with 256 double indirect blocks — 16G bytes 



Figure 4.7. Byte Capacity of a File - IK Bytes Per Block 



algorithm bmap /• block map of logical file byte offset to file system block */ 
input: (1) inode 

(2) byte offset 

output: (1) block number in file system 

(2) byte offset into block 

(3) bytes of I/O in block 

(4) read ahead block number 

{ 

calculate logical block number in file from byte offset; 
calculate start byte in block for I/O; /♦ output 2 V 

calculate number of bytes to copy to user; /* output 3 V 
check if read— ahead applicable, mark inode; /* output 4 ♦/ 
determine level of indirection; 
while (not at necessary level of indirection) 

calculate index into inode or indirect block from 
logical block number in file; 
get disk block number from inode or indirect block; 
release buffer from previous disk read, if any (algorithm brelse); 
if (no more levels of indirection) 
return (block number); 
read indirect disk block (algorithm bread); 

adjust logical block number in file according to level of indirection; 

} 



Figure 4.8. Conversion of Byte Offset to Block Number in File System 



9000 in the file. If a process wants to access byte offset 350,000 in the file, it must 
access a double indirect block, number 9156 in the figure. Since an indirect block 
has room for 256 block numbers, the first byte accessed via the double indirect 
block is byte number 272,384 (256K + 10K); byte number 350,000 in a file is 
therefore byte number 77,616 of the double indirect block. Since each single 
indirect block accesses 256K bytes, byte number 350,000 must be in the 0th single 
indirect block of the double indirect block — block number 331. Since each direct 
block in a single indirect block contains IK bytes, byte number 77,616 of a single 
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Figure 4.9. Block Layout of a Sample File and its Inode 



indirect block is in the 75th direct block in the single indirect block — block 
number 3333. Finally, byte number 350,000 in the file is at byte number 816 in 
block 3333. 

Examining Figure 4.9 more closely, several block entries in the inode are 0, 
meaning that the logical block entries contain no data. This happens if no process 
ever wrote data into the file at any byte offsets corresponding to those blocks and 
hence the block numbers remain at their initial value, 0. No disk space is wasted 
for such blocks. Processes can cause such a block layout in a file by using the Iseek 
and write system calls, as described in the next chapter. The next chapter also 
describes how the kernel takes care of read system calls that access such blocks. 

The conversion of a large byte offset, particularly one that is referenced via the 
triple indirect block, is an arduous procedure that could require the kernel to access 
three disk blocks in addition to the inode and data block. Even if the kernel finds 
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the blocks in the buffer cache, the operation is still expensive, because the kernel 
must make multiple requests of the buffer cache and may have to sleep awaiting 
locked buffers. How effective is the algorithm in practice? That depends on how 
the system is used and whether the user community and job mix are such that the 
kernel accesses large files or small files more frequently. It has been observed 
[Mullender 841, however, that most files on UNIX systems contain less than 10K 
bytes, and many contain less than IK bytes! 1 Since 10K bytes of a file are stored in 
direct blocks, most file data can be accessed with one disk access. So in spite of the 

fact that accessing large files is an expensive operation, accessing common-sized 
files is fast. 

Two extensions to the inode structure just described attempt to take advantage 
of file size characteristics. A major principle in the 4.2 BSD file system 
implementation [McKusick 84] is that the more data the kernel can access on the 
disk in a single operation, the faster file access becomes. That argues for having 
larger logical disk blocks, and the Berkeley implementation allows logical disk 
blocks of 4K or 8K bytes. But having larger block sizes on disk increases block 
fragmentation, leaving large portions of disk space unused. For instance, if the 
logical block size is 8K bytes, then a file of size 12K bytes uses 1 complete block 
and half of a second block. The other half of the second block (4K bytes) is 
wasted; no other file can use the space for data storage. If the sizes of files are 
such that the number of bytes in the last block of a file is uniformly distributed, 
then the average wasted space is half a block per file; the amount of wasted disk 
space can be as high as 45% for a file system with logical blocks of size 4K bytes 
[McKusick 84]. The Berkeley implementation remedies the situation by allocating 
a block fragment to contain the last data in a file. One disk block can contain 
fragments belonging to several files. An exercise in Chapter 5 explores some details 
of the implementation. 

The second extension to the classic inode structure described here is to store file 
data in the inode (see [Mullender 84]). By expanding the inode to occupy an 
entire disk block, a small portion of the block can be used for the inode structures 
and the remainder of the block can store the entire file, in many cases, or the end 
of a file otherwise. The main advantage is that only one disk access is necessary to 
get the inode and its data if the file fits in the inode block. 



1‘ For a 19,978 files, Mullender and Tannenbaum say that approximately 85% of the files 

were smaller than 8K bytes and that 48% were smaller than IK bytes. Although these percentages 
will vary from one installation to the next, they are representative of many UNIX systems. 
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4.3 DIRECTORIES 

Recall from Chapter 1 that directories are the files that give the file system its 
hierarchical structure; they play an important role in conversion of a file name to 
an inode number. A directory is a file whose data is a sequence of entries, each 
consisting of an inode number and the name of a file contained in the directory. A 
path name is a null terminated character string divided into separate components 
by the slash 07") character. Each component except the last must be the name of 
a directory, but the last component may be a non-directory file. UNIX System V 
restricts component names to a maximum of 14 characters; with a 2 byte entry for 
the inode number, the size of a directory entry is 16 bytes. 
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Figure 4.10. Directory Layout for /etc 

Figure 4.10 depicts the layout of the directory "etc”. Every directory contains 
the file names dot and dot-dot and whose inode numbers are those of the 
directory and its parent directory, respectively. The inode number of in "/etc" 
is located at offset 0 in the file, and its value is 83. The inode number of is 
located at offset 16, and its value is 2. Directory entries may be empty, indicated 
by an inode number of 0. For instance, the entry at address 224 in "/etc" is 
empty, although it once contained an entry for a file named "crash". The program 
ntkfs initializes a file system so that and of the root directory have the root 
inode number of the file system. 
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The kernel stores data for a directory just as it stores data for an ordinary file, 
using the inode structure and levels of direct and indirect blocks. Processes may 
read directories in the same way they read regular files, but the kernel reserves 
exclusive right to write a directory, thus insuring its correct structure. The access 
permissions of a directory have the following meaning: read permission on a 
directory allows a process to read a directory; write permission allows a process to 
create new directory entries or remove old ones (via the creat , mknod , link , and 
unlink system calls), thereby altering the contents of the directory; execute 
permission allows a process to search the directory for a file name (it is meaningless 
to execute a directory). Exercise 4.6 explores the difference between reading and 
searching a directory. 



4.4 CONVERSION OF A PATH NAME TO AN INODE 

The initial access to a file is by its path name, as in the open % chdir (change 
directory), or link system calls. Because the kernel works internally with inodes 
rather than with path names, it converts the path names to inodes to access files. 
The algorithm namei parses the path name one component at a time, converting 
each component into an inode based on its name and the directory being searched, 
and eventually returns the inode of the input path name (Figure 4.1 1). 

Recall from Chapter 2 that every process is associated with (resides in) a 
current directory; the u area contains a pointer to the current directory inode. The 
current directory of the first process in the system, process 0, is the root directory. 
The current directory of every other process starts out as the current directory of its 
parent process at the time it was created (see Section 5.10). Processes change their 
current directory by executing the chdir (change directory) system call. All path 
name searches start from the current directory of the process unless the path name 
starts with the slash character, signifying that the search should start from the root 
directory. In either case, the kernel can easily find the inode where the path name 
search starts: The current directory is stored in the process u area , and the system 
root inode is stored in a global variable. 2 

Namei uses intermediate inodes as it parses a path name; call them working 
inodes. The inode where the search starts is the first working inode. During each 
iteration of the namei loop, the kernel makes sure that the working inode is indeed 
that of a directory. Otherwise, the system would violate the assertion that non- 
directory files can only be leaf nodes of the file system tree. The process must also 
have permission to search the directory (read permission is insufficient). The user 
ID of the process must match the owner or group ID of the file, and execute 



2. A process can execute the chroot system call to change its notion of the file system root. The 
changed root is stored in the u area. 
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algorithm namei /* convert path name to inode */ 
input: path name 
output: locked inode 

I 

if (path name starts from root) 

working inode — root inode (algorithm iget); 

else 

working inode — current directory inode (algorithm iget): 

while (there is more path name) 

( 

read next path name component from input; 

verify that working inode is of directory, access permissions OK; 

if (working inode is of root and component is "..") 

continue; /* loop back to while V 
read directory (working inode) by repeated use of algorithms 
bmap, bread and brelse; 

if (component matches an entry in directory (working inode)) 

( 

get inode number for matched component; 
release working inode (algorithm iput); 

working inode — inode of matched component (algorithm iget); 

} 

else /• component not in directory V 
return (no inode); 

} 

return (working inode); 

} 



Figure 4.11. Algorithm for Conversion of a Path Name to an Inode 



permission must be granted, or the file must allow search to all users. Otherwise 
the search fails. 

The kernel does a linear search of the directory file associated with the working 
inode, trying to match the path name component to a directory entry name. 
Starting at byte offset 0, it converts the byte offset in the directory to the 
appropriate disk block according to algorithm bmap and reads the block using 
algorithm bread. It searches the block for the path name component, treating the 
contents of the block as a sequence of directory entries. If it finds a match, it 
records the inode number of the matched directory entry, releases the block 
(algorithm brelse ) and the old working inode (algorithm iput), and allocates the 
inode of the matched component (algorithm iget). The new inode becomes the 
working inode. If the kernel does not match the path name with any names in the 
block, it releases the block, adjusts the byte offset by the number of bytes in a 
block, converts the new offset to a disk block number (algorithm bmap), and reads 
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the next block. The kernel repeats the procedure until it matches the path name 
component with a directory entry name, or until it reaches the end of the directory. 

For example, suppose a process wants to open the file “/etc/pass wd”. When the 
kernel starts parsing the file name, it encounters “/” and gets the system root 
inode. Making root its current working inode, the kernel gathers in the string 
“etc”. After checking that the current inode is that of a directory (“/”) and that 
the process has the necessary permissions to search it, the kernel searches root for a 
file whose name is “etc”: It accesses the data in the root directory block by block 
and searches each block one entry at a time until it locates an entry for “etc”. On 
finding the entry, the kernel releases the inode for root (algorithm iput) and 
allocates the inode for “etc” (algorithm iget) according to the inode number of the 
entry just found. After ascertaining that “etc” is a directory and that it has the 
requisite search permissions, the kernel searches “etc” block by block for a 
directory structure entry for the file “passwd”. Referring to Figure 4.10, it would 
find the entry for “passwd” as the ninth entry of the directory. On finding it, the 
kernel releases the inode for “etc”, allocates the inode for “passwd”, and — since 
the path name is exhausted — returns that inode. 

It is natural to question the efficiency of a linear search of a directory for a path 
name component. Ritchie points out (see page 1968 of [Ritchie 78b]) that a linear 
search is efficient because it is bounded by the size of the directory. Furthermore, 
early UNIX system implementations did not run on machines with large memory 
space, so there was heavy emphasis on simple algorithms such as linear search 
schemes. More complicated search schemes could require a different, more 
complex, directory structure, and would probably run more slowly on small 
directories than the linear search scheme. 



4.5 SUPER BLOCK 

So far, this chapter has described the structure of a file, assuming that the inode 
was previously bound to a file and that the disk blocks containing the data were 
already assigned. The next sections cover how the kernel assigns inodes and disk 
blocks. To understand those algorithms, let us examine the structure of the super 
block. 

The super block consists of the following fields: 

• the size of the file system, 

• the number of free blocks in the file system, 

• a list of free blocks available on the file system, 

• the index of the next free block in the free block list, 

• the size of the inode list, 

• the number of free inodes in the file system, 

• a list of free inodes in the file system, 

• the index of the next free inode in the free inode list. 
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• lock fields for the free block and free inode lists, 

• a flag indicating that the super block has been modified. 

The remainder of this chapter will explain the use of the arrays, indices and locks. 
The kernel periodically writes the super block to disk if it had been modified so that 
it is consistent with the data in the file system. 



4.6 INODE ASSIGNMENT TO A NEW FILE 

The kernel uses algorithm iget to allocate a known inode, one whose (file system 
and) inode number was previously determined. In algorithm nantei for instance, 
the kernel determines the inode number by matching a path name component to a 
name in a directory. Another algorithm, ialloc, assigns a disk inode to a newly 
created file. 

The file system contains a linear list of inodes, as mentioned in Chapter 2. An 
inode is free if its type field is zero. When a process needs a new inode, the kernel 
could theoretically search the inode list for a free inode. However, such a search 
would be expensive, requiring at least one read operation (possibly from disk) for 
every inode. To improve performance, the file system super block contains an array 
to cache the numbers of free inodes in the file system. 

Figure 4.12 shows the algorithm ialloc for assigning new inodes. For reasons 
cited later, the kernel first verifies that no other processes have locked access to the 
super block free inode list. If the list of inode numbers in the super block is not 
empty, the kernel assigns the next inode number, allocates a free in-core inode for 
the newly assigned disk inode using algorithm iget (reading the inode from disk if 
necessary), copies the disk inode to the in-core copy, initializes the fields in the 
inode, and returns the locked inode. It updates the disk inode to indicate that the 
inode is now in use: A non-zero file type field indicates that the disk inode is 
assigned. In the simplest case, the kernel has a good inode, but race conditions 
exist that necessitate more checking, as will be explained shortly. Loosely defined, 
a race condition arises when several processes alter common data structures such 
that the resulting computations depend on the order in which the processes 
executed, even though all processes obeyed the locking protocol. For example, it is 
implied here that a process could get a used inode. A race condition is related to 
the mutual exclusion problem defined in Chapter 2, except that locking schemes 
solve the mutual exclusion problem there but may not, by themselves, solve all race 
conditions. 

If the super block list of free inodes is empty, the kernel searches the disk and 
places as many free inode numbers as possible into the super block. The kernel 
reads the inode list on disk, block by block, and fills the super block list of inode 
numbers to capacity, remembering the highest-numbered inode that it finds. Call 
that inode the “remembered” inode; it is the last one saved in the super block. The 
next time the kernel searches the disk for free inodes, it uses the remembered inode 
as its starting point, thereby assuring that it wastes no time reading disk blocks 
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algorithm lalloc /* allocate inode */ 
input: file system 

output: locked inode 

{ 

while (not done) 

{ 

if (super block locked) 

( 

sleep (event super block becomes free); 
continue; /* while loop V 

if (inode list in super block is empty) 

lock super block; 

get remembered inode for free inode search; 
search disk for free inodes until super block full, 

or no more free inodes (algorithms bread and brclsc); 
unlock super block; 

wake up (event super block becomes free); 
if (no free inodes found on disk) 
return (no inode); 

set remembered inode for next free inode search; 

/• there are inodes in super block inode list */ 
get inode number from super block inode list; 
get inode (algorithm iget); 
if (inode not free after all) /* !!! V 
{ 



write inode to disk; 

release inode (algorithm iput); 

continue; /* while loop V 

/* inode is free V 
initialize inode; 
write inode to disk; 

decrement file system free inode count; 
return (inode); 

) 

j 



Figure 4.12. Algorithm for Assigning New Inodes 
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where no free inodes should exist. After gathering a fresh set of free inode 
numbers, it starts the inode assignment algorithm from the beginning. Whenever 
the kernel assigns a disk inode, it decrements the free inode count recorded in the 
super block. 
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Figure 4.13. Two Arrays of Free Inode Numbers 
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Consider the two pairs of arrays of free inode numbers in Figure 4.13. If the 
list of free inodes in the super block looks like the first array in Figure 4.13(a) 
when the kernel assigns an inode, it decrements the index for the next valid inode 
number to 18 and takes inode number 48. If the list of free inodes in the super 
block looks like the first array in Figure 4.13(b), it will notice that the array is 
empty and search the disk for free inodes, starting from inode number 470, the 
remembered inode. When the kernel fills the super block free list to capacity, it 
remembers the last inode as the start point for the next search of the disk. The 
kernel assigns an inode it just took from the disk (number 471 in the figure) and 
continues whatever it was doing. 



algorithm ifree /* inode free V 
input: file system inode number 

output: none 

{ 

increment file system free inode count; 
if (super block locked) 
return; 

if (inode list full) 

{ 

if (inode number less than remembered inode for search) 

set remembered inode for search * input inode number; 

) 

else 

store inode number in inode list; 
return; 

1 



Figure 4.14. Algorithm for Freeing Inode 



The algorithm for freeing an inode is much simpler. After incrementing the 
total number of available inodes in the file system, the kernel checks the lock on the 
super block. If locked, it avoids race conditions by returning immediately: The 
inode number is not put into the super block, but it can be found on disk and is 
available for reassignment. If the list is not locked, the kernel checks if it has room 
for more inode numbers and, if it does, places the inode number in the list and 
returns. If the list is full, the kernel may not save the newly freed inode there: It 
compares the number of the freed inode with that of the remembered inode. If the 
freed inode number is less than the remembered inode number, it “remembers” the 
newly freed inode number, discarding the old remembered inode number from the 
super block. The inode is not lost, because the kernel can find it by searching the 
inode list on disk. The kernel maintains the super block list such that the last inode 
it dispenses from the list is the remembered inode. Ideally, there should never be 
free inodes whose inode number is less than the remembered inode number, but 
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Figure 4.15. Placing Free Inode Numbers into the Super Block 



exceptions are possible. 

Consider two examples of freeing inodes. If the super block list of free inodes 
has room for more free inode numbers as in Figure 4.13(a), the kernel places the 
inode number on the list, increments the index to the next free inode, and proceeds. 
But if the list of free inodes is full as in Figure 4.15, the kernel compares the inode 
number it has freed to the remembered inode number that will start the next disk 
search. Starting with the free inode list in Figure 4.15(a), if the kernel frees inode 
499, it makes 499 the remembered inode and evicts number 535 from the free list. 
If the kernel then frees inode number 601, it does not change the contents of the 
free list. When it later uses up the inodes in the super block free list, it will search 
the disk for free inodes starting from inode number 499, and find inodes 535 and 
601 again. 
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Figure 4.16. Race Condition in Assigning Inodes 



The preceding paragraph described the simple cases of the algorithms. Now 
consider the case where the kernel assigns a new inode and then allocates an in-core 
copy for the inode. The algorithm implies that the kernel could find that the inode 
had already been assigned. Although rare, the following scenario shows such a case 
(refer to Figures 4.16 and 4.17). Consider three processes. A, B, and C, and 
suppose that the kernel, acting on behalf of process A, 3 assigns inode I but goes to 
sleep before it copies the disk inode into the in-core copy. Algorithms iget (invoked 



3. As in the last chapter, the term "process” here will mean "the kernel, acting on behalf of a process.” 
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Figure 4.17. Race Condition in Assigning Inodes (continued) 



by ialloc) and bread (invoked by iget) give process A ample opportunity to go to 
sleep. While process A is asleep, suppose process B attempts to assign a new inode 
but discovers that the super block list of free inodes is empty. Process B searches 
the disk for free inodes, and suppose it starts its search for free inodes at an inode 
number lower than that of the inode that A is assigning. It is possible for process 
B to find inode I free on the disk since process A is still asleep, and the kernel does 
not know that the inode is about to be assigned. Process B, not realizing the 
danger, completes its search of the disk, fills up the super block with (supposedly) 
free inodes, assigns an inode, and departs from the scene. However, inode I is in 
the super block free list of inode numbers. When process A wakes up, it completes 
the assignment of inode I. Now suppose process C later requests an inode and 
happens to pick inode I from the super block free list. When it gets the in-core 
copy of the inode, it will find its file type set, implying that the inode was already 
assigned. The kernel checks for this condition and, finding that the inode has been 
assigned, tries to assign a new one. Writing the updated inode to disk immediately 
after its assignment in ialloc makes the chance of the race smaller, because the file 
type field will mark the inode in use. 
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Locking the super block list of inodes while reading in a new set from disk 
prevents other race conditions. If the super block list were not locked, a process 
could find it empty and try to populate it from disk, occasionally sleeping while 
waiting for I/O completion. Suppose a second process also tried to assign a new 
inode and found the list empty. It, too, would try to populate the list from disk. 
At best, the two processes are duplicating their efforts and wasting CPU power. At 
worst, race conditions of the type described in the previous paragraph would be 
more frequent. Similarly, if a process freeing an inode did not check that the list is 
locked, it could overwrite inode numbers already in the free list while another 
process was populating it from disk. Again, the race conditions described above 
would be more frequent. Although the kernel handles them satisfactorily, system 
performance would suffer. Use of the lock on the super block free list prevents 
such race conditions. 



4.7 ALLOCATION OF DISK BLOCKS 

When a process writes data to a file, the kernel must allocate disk blocks from the 
file system for direct data blocks and, sometimes, for indirect blocks. The file 
system super block contains an array that is used to cache the numbers of free disk 
blocks in the file system. The utility program mkfs (make file system) organizes 
the data blocks of a file system in a linked list, such that each link of the list is a 
disk block that contains an array of free disk block numbers, and one array entry is 
the number of the next block of the linked list. Figure 4.18 shows an example of 
the linked list, where the first block is the super block free list and later blocks on 
the linked list contain more free block numbers. 

When the kernel wants to allocate a block from a file system (algorithm alloc , 
Figure 4.19), it allocates the next available block in the super block list. Once 
allocated, the block cannot be reallocated until it becomes free. If the allocated 
block is the last available block in the super block cache, the kernel treats it as a 
pointer to a block that contains a list of free blocks. It reads the block, populates 
the super block array with the new list of block numbers, and then proceeds to use 
the original block number. It allocates a buffer for the block and clears the buffer’s 
data (zeros it). The disk block has now been assigned, and the kernel has a buffer 
to work with. If the file system contains no free blocks, the calling process receives 
an error. 

If a process writes a lot of data to a file, it repeatedly asks the system for blocks 
to store the data, but the kernel assigns only one block at a time. The program 
mkfs tries to organize the original linked list of free block numbers so that block 
numbers dispensed to a file are near each other. This helps performance, because it 
reduces disk seek time and latency when a process reads a file sequentially. Figure 
4.18 depicts block numbers in a regular pattern, presumably based on the disk 
rotation speed. Unfortunately, the order of block numbers on the free block linked 
lists breaks down with heavy use as processes write files and remove them, because 
block numbers enter and leave the free list at random. The kernel makes no 
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Figure 4.18. Linked List of Free Disk Block Numbers 



attempt to sort block numbers on the free list. 

The algorithm free for freeing a block is the reverse of the one for allocating a 
block. If the super block list is not full, the block number of the newly freed block 
is placed on the super block list. If, however, the super block list is full, the newly 
freed block becomes a link block; the kernel writes the super block list into the 
block and writes the block to disk. It then places the block number of the newly 
freed block in the super block list: That block number is the only member of the 
list. 

Figure 4.20 shows a sequence of alloc and free operations, starting with one 
entry on the super block free list. The kernel frees block 949 and places the block 
number on the free list. It then allocates a block and removes block number 949 
from the free list. Finally, it allocates a block and removes block number 109 from 
the free list. Because the super block free list is now empty, the kernel replenishes 
the list by copying in the contents of block 109, the next link on the linked list. 
Figure 4.20(d) shows the full super block list and the next link block, block 211. 

The algorithms for assigning and freeing inodes and disk blocks are similar in 
that the kernel uses the super block as a cache containing indices of free resources, 
block numbers, and inode numbers. It maintains a linked list of block numbers 
such that every free block number in the file system appears in some element of the 
linked list, but it maintains no such list of free inodes. There are three reasons for 
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algorithm alloc /* file system block allocation •/ 
input: file system number 
output: buffer for new block 
{ 

while (super block locked) 

sleep (event super block not locked); 
remove block from super block free list; 
if (removed last block from free list) 

( 

lock super block; 

read block just taken from free list (algorithm bread); 
copy block numbers in block into super block; 
release block buffer (algorithm brelse); 
unlock super block; 

wake up processes (event super block not locked); 

) 

get buffer for block removed from super block list (algorithm getblk); 

zero buffer contents; 

decrement total count of free blocks; 

mark super block modified; 

return buffer; 

) 



Figure 4.19. Algorithm for Allocating Disk Block 



the different treatment. 

1. The kernel can determine whether an inode is free by inspection: If the file 
type field is clear, the inode is free. The kernel needs no other mechanism to 
describe free inodes. However, it cannot determine whether a block is free 
just by looking at it. It could not distinguish between a bit pattern that 
indicates the block is free and data that happened to have that bit pattern. 
Hence, the kernel requires an external method to identify free blocks, and 
traditional implementations have used a linked list. 

2. Disk blocks lend themselves to the use of linked lists: A disk block easily 
holds large lists of free block numbers. But inodes have no convenient place 
for bulk storage of large lists of free inode numbers. 

3. Users tend to consume disk block resources more quickly than they consume 
inodes, so the apparent lag in performance when searching the disk for free 
inodes is not as critical as it would be for searching for free disk blocks. 
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(a) Original configuration 




(b) After freeing block number 949 




(c) After assigning block number (949) 




(d) After assigning block number (109) 
replenish super block free list 



Figure 4.20. Requesting and Freeing Disk Blocks 
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4.8 OTHER FILE TYPES 

The UNIX system supports two other file types: pipes and special files. A pipe, 
sometimes called a fifo (for ‘‘first-in-first-out”), differs from a regular file in that its 
data is transient: Once data is read from a pipe, it cannot be read again. Also, the 
data is read in the order that it was written to the pipe, and the system allows no 
deviation from that order. The kernel stores data in a pipe the same way it stores 
data in an ordinary file, except that it uses only the direct blocks, not the indirect 
blocks. The next chapter will examine the implementation of pipes. 

The last file types in the UNIX system are special files, including block device 
special files and character device special files. Both types specify devices, and 
therefore the file inodes do not reference any data. Instead, the inode contains two 
numbers known as the major and minor device numbers. The major number 
indicates a device type such as terminal or disk, and the minor number indicates 
the unit number of the device. Chapter 10 examines special devices in detail. 



4.9 SUMMARY 

The inode is the data structure that describes the attributes of a file, including the 
layout of its data on disk. There are two versions of the inode: the disk copy that 
stores the inode information when the file is not in use and the in-core copy that 
records information about active files. Algorithms ialloc and ifree control 
assignment of a disk inode to a file during the create mknod , pipe , and unlink 
system calls (next chapter), and the algorithms iget and iput control the allocation 
of in-core inodes when a process accesses a file. Algorithm bmap locates the disk 
blocks of a file, according to a previously supplied byte offset in the file. Directories 
are files that correlate file name components to inode numbers. Algorithm namei 
converts file names manipulated by processes to inodes, used internally by the 
kernel. Finally, the kernel controls assignment of new disk blocks to a file using 
algorithms alloc and free . 

The data structures discussed in this chapter consist of linked lists, hash queues, 
and linear arrays, and the algorithms that manipulate the data structures are 
therefore simple. Complications arise due to race conditions caused by the 
interaction of the algorithms, and the text has indicated some of these timing 
problems. Nevertheless, the algorithms are not elaborate and illustrate the 
simplicity of the system design. 

The structures and algorithms explained here are internal to the kernel and are 
not visible to the user. Referring to the overall system architecture (Figure 2.1), 
the algorithms described in this chapter occupy the lower half of the file subsystem. 
The next chapter examines the system calls that provide the user interface to the 
file system, and it describes the upper half of the file subsystem that invokes the 
internal algorithms described here. 
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4.10 EXERCISES 

1. The C language convention counts array indices from 0. Why do inode numbers start 
from 1 and not 0? 

2. If a process sleeps in algorithm iget when it finds the inode locked in the cache, why 
must it start the loop again from the beginning after waking up? 

3. Describe an algorithm that takes an in-core inode as input and updates the 
corresponding disk inode. 

4. The algorithms iget and iput do not require the processor execution level to be raised 
to block out interrupts. What does this imply? 

5. How efficiently can the loop for indirect blocks in bmap be encoded? 

mkdir junk 

for i in 1 2 3 4 5 

do 

echo hello > junk/Si 
done 

Is —Id junk 
Is —1 junk 
chmod — r junk 
Is —Id junk 
Is junk 
Is —I junk 
cd junk 
pwd 
Is —1 
echo * 
cd .. 

chmod +r junk 
chmod —x junk 
Is junk 
Is —1 junk 
cd junk 

chmod +x junk 



Figure 4.21. Difference between Read and Search Permission on Directories 



6. Execute the shell command script in Figure 4.21. It creates a directory “junk” and 
creates five files in the directory. After doing some control Is commands, the chmod 
command turns off read permission for the directory. What happens when the various 
Is commands are executed now? What happens after changing directory into “junk”? 
After restoring read permission but removing execute (search) permission from “junk”, 
repeat the experiment. What happens? What is happening in the kernel to cause this 
behavior? 

7. Given the current structure of a directory entry on a System V system, what is the 
maximum number of files a file system can contain? 
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8. UNIX System V allows a maximum of 14 characters for a path name component. 
Namei truncates extra characters in a component. How should the file system and 
respective algorithms be redesigned to allow arbitrary length component names? 

9. Suppose a user has a private version of the UNIX system but changes it so that a path 
name component can consist of 30 characters; the private version of the operating 
system stores the directory entries the same way that the standard operating system 
does, except that the directory entries are 32 bytes long instead of 16. If the user 
mounts the private file system on a standard system, what would happen in algorithm 
namei when a process accesses a file on the private file system? 

*10. Consider the algorithm namei for converting a path name into an inode. As the search 
progresses, the kernel checks that the current working inode is that of a directory. Is 
it possible for another process to remove ( unlink ) the directory? How can the kernel 
prevent this? The next chapter will come back to this problem. 

*11. Design a directory structure that improves the efficiency of searching for path names 
by avoiding the linear search. Consider two techniques: hashing and w-ary trees. 

* 12. Design a scheme that reduces the number of directory searches for file names by 

caching frequently used names. 

• 13. Ideally, a file system should never contain a free inode whose inode number is less than 

the “remembered” inode used by ialloc. How is it possible for this assertion to be 
false? 

14. The super block is a disk block and contains other information besides the free block 
list, as described in this chapter. Therefore, the super block free list cannot contain as 
many free block numbers as can be potentially stored in a disk block on the linked list 
of free disk blocks. What is the optimal number of free block numbers that should be 
stored in a block on the linked list? 

*15. Discuss a system implementation that keeps track of free disk blocks with a bit map 
instead of a linked list of blocks. What are the advantages and disadvantages of this 
scheme? 
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SYSTEM CALLS 
FOR THE FILE SYSTEM 



The last chapter described the internal data structures for the file system and the 
algorithms that manipulate them. This chapter deals with system calls for the file 
system, using the concepts explored in the previous chapter. It starts with system 
calls for accessing existing files, such as open, ready write , lseek y and close , then 
presents system calls to create new files, namely, creat and mknod , and then 
examines the system calls that manipulate the inode or that maneuver through the 
file system: chdir t chroot , chown , chmod , stat y and fstat. It investigates more 
advanced system calls: pipe and dup are important for the implementation of pipes 
in the shell; mount and umount extend the file system tree visible to users; link and 
unlink change the structure of the file system hierarchy. Then, it presents the 
notion of file system abstractions, allowing the support of various file systems as 
long as they conform to standard interfaces. The last section in the chapter covers 
file system maintenance. The chapter introduces three kernel data structures: the 
file table, with one entry allocated for every opened file in the system, the user file 
descriptor table, with one entry allocated for every file descriptor known to a 
process, and the mount table, containing information for every active file system. 

Figure 5.1 shows the relationship between the system calls and the algorithms 
described previously. It classifies the system calls into several categories, although 
some system calls appear in more than one category: 
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File System Calls 



1 —Mill 


Return 

File 

Desc 


Use of 
namei 


Assign 

inodes 


File 

Attributes 


File 

I/O 


File Sys 
Structure 


Tree 

Manipulation 


open 

creat 

dup 

pipe 

close 


open stat 
creat link 
chdir unlink 
chroot mknod 
chown mount 
chmod umount 


creat 

mknod 

link 

unlink 


chown 

chmod 

stat 


read 

write 

Iseek 


mount 

umount 


chdir 

chown 



Lower Level File System Algorithms 



namei 






iget iput 


ialloc ifree 


alloc free bmap 




buffer allocation algorithms 


getblk 


brelse bread 


breada bwrite 



Figure 5.1. File System Calls and Relation to Other Algorithms 



• System calls that return file descriptors for use in other system calls; 

• System calls that use the namei algorithm to parse a path name; 

• System calls that assign and free inodes, using algorithms ialloc and ifree\ 

• System calls that set or change the attributes of a file; 

• System calls that do I/O to and from a process, using algorithms alloc , free , 
and the buffer allocation algorithms; 

• System calls that change the structure of the file system; 

• System calls that allow a process to change its view of the file system tree. 



5.1 OPEN 

The open system call is the first step a process must take to access the data in a 
file. The syntax for the open system call is 

fd * open (pathname, flags, modes); 

where pathname is a file name, flags indicate the type of open (such as for reading 
or writing), and modes give the file permissions if the file is being created. The 
open system call returns an integer 1 called the user file descriptor. Other file 





